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ABSTRACT 


The  application  of  virtual  environments  to  the  product  design  process  has 
intrigued  researchers  and  practitioners  in  industry,  government,  and  academia.  Using 
virtual  environment  tools,  design  teams  can  evaluate  future  designs  in  synthetic 
environments  before  building  physical  prototypes,  thus  creating  a  unique  capability  that 
reduces  cost,  shrinks  time  to  market,  and  improves  product  quality.  However,  a 
proliferation  of  visual  display  devices  used  to  generate.  3-D  stereoscopic  information  is 
forcing  organizations  to  decide  which  tools  to  use  to  enhance  product  design  activities. 

This  study  evaluated,  through  comparative  empirical  testing  and  data  analysis, 
how  several  commercially-available  virtual  3-D  visual  display  systems  contribute  to 
cross-functional  team  collaboration  in  a  product  design  review.  A  4x4  Graeco-Latin 
Square  experimental  design  assessed  the  value  of  a  Helmet  Mounted  Display,  a  Binocular 
Omni-Orientation  Monitor,  stereoscopic  glasses  with  monitor,  and  traditional 
monoscopic  CRT  monitor  technologies  for  use  in  a  concept  design  review  process.  The 
experiment  was  conducted  using  personnel  from  several  functional  elements  of  the  U.  S. 
Army  Tank-automotive  and  Armaments  Command.  Team  performance  measures, 
questionnaires,  and  observer  evaluation  aided  the  assessment  of  tested  display  systems. 

Empirical  evaluations  revealed  that  design  teams  detected  more  errors  when 
using  the  stereoscopic  glasses  and  monoscopic  CRT  monitor  systems,  detected  errors 
fastest  when  using  the  HMD  system,  and  found  no  differences  between  the  display 


systems  for  the  time  to  resolve  design  problems.  Subjective  data  were  used  to  evaluate 
participant  perceptions  of  each  technology  and  user  preferences. 

Based  on  these  findings,  the  best  method  for  integrating  virtual  3-D  displays  is  a 
combined  technology  approach.  It  begins  with  clearly  defined  design  review  objectives, 
individual  member  evaluation  of  the  proposed  design  using  a  HMD  device,  and  team 
evaluation  using  stereoscopic  glasses  to  stimulate  cross-discussion  and  identification  of 
design  errors.  Lastly,  natural  face-to-face  communication  should  be  used  for  teams  to 
resolve  design  problems.  This  study’s  results  strengthen  the  premise  that  virtual 
environments  improve  the  effectiveness  and  efficiency  of  the  design  review  process. 
They  also  lead  to  a  better  understanding  of  the  trade-off  required  when  selecting  a  visual 
display  system  and  provide  insight  into  future  virtual  environment  interface  designs. 
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CHAPTER  1 


INTRODUCTION 

The  traditional  product  development  process  can  be  described  as  a  sequential, 
linear  process.  Phases  include  concept  development,  system-level  design,  detail  design, 
testing  and  refinement,  and  production  ramp-up  (Ulrich  &  Eppinger,  1995).  Shu  and 
Bowers  (1994)  characterize  this  activity  as  a  sequence  of  "throw  it  over  the  wall" 
processes  where  functional  groups  make  contributions  to  a  project  sometimes 
independent  of  other  elements.  As  a  result,  critical  decisions  that  can  significantly  impact 
overall  product  design  and  development  are  frequently  made  without  regard  to  the  effect 
on  and  consequences  to  other  functional  elements. 

Some  process  improvements  are  evolving.  The  need  to  shorten  the  development 
cycle  (Cooper  &  Kleinshmidt,  1994;  Cordero,  1991;  Iansiti,  1993;  Vesey,  1991),  to 
achieve  higher  quality  (Ali,  Krapfel,  &  LaBahn,  1995;  Cordero,  1991),  and  to  solicit 
direct  customer  feedback  (Larson -Mogal,  1994)  have  stimulated  the  need  for  changes  to 
traditional  product  development  methods.  As  a  result,  numerous  organizations  are 
investigating  the  use  of  concurrent  cross-functional  teams  coupled  with  emerging 
computer  technologies  to  create  more  robust,  collaborative  virtual  design  environments. 

This  chapter  provides  an  overview  of  a  contemporary  virtual  environment-based 
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design  review  process;  a  transition  in  methodologies.  Also  it  addresses  the  need  for  the 
research  study,  identifies  statements  of  problems,  and  defines  terms  used  in  the  research. 

Overview  of  the  Contemporary  Product  Design  Environment 

In  the  past,  products  were  designed  on  drawing  boards  using  pencils  and  two- 
dimensional  (2-D)  paper  drawings.  The  product  design  process,  using  these  drawings, 
involved  serial  reviews  by  various  functional  representatives,  and  manual  drawing 
revisions  —  a  very  time  intensive  process.  Once  designs  were  complete,  product 
manufacturers  often  created  physical  or  functional  prototypes  so  design  teams  (and 
maybe  their  customers)  could  see,  touch,  and  generally  experience  a  product  (Larson- 
Mogal,  1994). 

The  onset  of  computer  technology  has  had  a  positive  impact  on  the  product  design 
process,  which  has  caused  a  paradigm  shift  in  the  way  products  are  designed.  Today,  the 
process  is  initiated  by  developing  2-D  and  three-dimensional  (3-D)  solid  models  of 
alternative  designs  on  Computer  Aided  Design  (CAD)  stations  -  sometimes  saving  up  to 
70%  of  total  manual  design  time  (Barfield,  Chang,  Majchrzak,  Eberts,  &  Salvendy, 

1987).  This  technology  has  provided  expanded  capabilities  for  the  development  and 
timely  revision  of  new  product  designs.  However,  while  these  enhanced  the  productivity 
of  a  single  designer,  their  use  is  not  very  effective  in  a  collaborative,  concurrent  design  or 
team  review  environment.  To  overcome  this  limitation,  new  tools  and  approaches  are 
needed. 
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Emerging  technologies  that  have  the  potential  to  significantly  impact  the  product 
design  process  are  virtual  environment  (VE)  systems.  By  their  nature,  these  systems  are 
capable  of  stimulating  the  human  senses  of  sight,  sound,  and  touch.  They  allow  a  person 
to  experience  life-like  domains  and  objects  that  appear  to  be  real  but  only  exist  in  a 
computer-based  environment.  In  the  eyes  of  a  designer,  immersion  in  such  a  near-realistic 
environment  can  provide  visualization  of  the  final  product  from  various  perspectives. 
Figure  1  depicts  the  described  evolution  of  product  design  capabilities.  As  is  shown, 
design  has  evolved  from  traditional  2-D  hand  drawings  to  3-D  CAD  on  flat  screen 
computer  CRT  monitors.  Beyond  flat  screen  technologies  are  individual  and 
collaborative  stereoscopic  VE  systems.  The  use  of  VEs  is  projected  to  be  the  next  major 
step  in  the  product  design  evolution  and  that  movement  will  incorporate  collaborative 
design  environments  (Gadh,  1994;  Gottschalk  &  Machlis,  1994;  Kalawsky,  1993; 

Kitfield,  1994;  Larson-Mogal,  1994;  Pratt,  1994). 

Potential  benefits  exist  in  VEs  capabilities  to  allow  product  design  team  members 
to  simultaneously  view  a  virtual  product,  and  jointly  evaluate  design  issues,  ideas,  and 
parameters  from  various  viewpoints.  These  visualization  techniques  have  the  potential  to 
clarify  information  and  enhance  collaboration  between  representatives  within  an 
organization  at  various  levels  to  make  more  informed  decisions  which  can  result  in  lower 
final  development  costs. 
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Figure  1.  Design  Evolution 

Some  work  has  explored  the  benefits  of  3-D  displays  for  a  variety  of 
applications.  Evaluating  pilot  operations,  Sanders  and  McCormick  ( 1 993)  found  that 
pilot  decision  times  were  faster  when  using  3-D  displays.  In  another  study,  Wichansky 
(1991)  concluded  that  3-D  stereo  views  are  better  suited  for  applications  involving 
viewing  of  complex  and  abstract  objects.  Similarly,  McWhorter,  Hodges,  and  Rodriguez 
(1991)  determined  that  3-D  stereoscopic  displays  were  superior  for  visual  search  tasks 
and  for  spatial  judgment  tasks.  Wickens  (1992)  contends  that  3-D  displays  should  be  used 
to  represent  3-D  information  because  they  are  more  compatible  with  the  operator’s 
mental  model  of  the  3-D  world  then  traditional  2-D  displays. 

There  are,  however,  several  important  challenges  that  exist  in  integrating  VE 
visualization  technologies  into  the  product  design  and  evaluation  process.  Issues  include 
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determining  appropriate  3-D  visual  display  devices  and  managing  the  simultaneous  use 
of  these  technologies  by  multi-functional  design  teams. 

Need  for  the  Study 

Global  competition,  shortened  product  lives,  rapid  technology  growth,  declining 
resources,  flattening  of  organizations,  downsizing,  loss  of  valued  employees  due  to 
attrition,  and  shrinking  budgets  are  forcing  organizations  to  do  more  with  less.  Improved 
product  design  processes  are  required  if  organizations  expect  the  same  or  improved 
effectiveness  and  efficiency  from  current  product  design  and  development  practices. 
While  product  development  organizations  recognize  the  need  to  improve  the  product 
design  process  and  realize  that  cross-functional  collaboration  and  computer  technology 
are  keys  to  achieving  that  goal,  they  do  not  fully  understand  how  to  integrate  the  two  into 
a  single  collaborative  virtual  design  environment.  There  is  a  need  to  gain  insight  into 
these  tools  and  processes  in  order  to  develop  improved  design  practices. 

Decisions  made  in  the  early  stages  of  product  design  effect  time  to  market  and 
product  cost.  A  National  Research  Council  report  on  National  Design  Strategy  concluded 
that  85%  of  product  development  costs  are  determined  before  the  product  design  is 
released  to  manufacturing  (Will,  1991).  For  this  reason  alone,  there  is  a  need  to  develop 
tools  that  will  help  organizations  make  better,  more  informed  decisions  early  in  the 
design  process  thereby  reducing  cost,  shrinking  development  time,  and  eliminating 
unnecessary  risks  associated  with  traditional  product  development  methods. 
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Many  in  the  VE  community  believe  that  the  use  of  a  collaborative  virtual  design 
environment  will  have  a  positive  impact  on  the  effectiveness  and  efficiency  of  the 
product  design  process  (Ashley,  1995;  Hedberg,  1994;  Keebler,  1993;  National  Research 
Council,  1995).  They  believe  that  collaborative  virtual  environments  have  the  capability 
to  promote  communication  interchanges  across  all  individual  organizational  functions. 
The  voice  of  the  virtual  design  community,  as  articulated  by  Ellis  (1997),  echoes  the 
belief  that  VEs  are  powerful  communication  tools.  Indicating  that  enhanced  visualization 
has  solved  some  fundamental  communication  problems  for  General  Motors  design  teams, 
which  included  people  with  diverse  backgrounds  and  skills  —  designers,  engineers, 
manufacturing  engineers.  Ellis  (1997)  quotes  an  interview  with  Randy  Smith  of  GM’s 
R  &  D  Center,  where  he  said: 

It  turns  out  that  in  many  cases  these  people  don’t  understand  a  simple  model,  like 
a  drawing  on  the  wall,  in  the  same  way  and  they  may  not  all  understand  a  color 
picture  on  a  flat  graphics  tube  in  the  same  way.  There’s  still  some  subjective 
interpretation.  But  give  them  a  full-scale  model  with  depth  that  is  as  close  as  we 
get  can  get  these  days  to  physical  parts,  and  they  can  all  communicate  about  it  in 
the  same  way;  they  all  understand  it  immediately.  So  this  becomes  a  very 
powerful  communication  tool.  (p.  32) 

Enhanced  visualization  allows  design  team  members  to  simultaneously  enter  a 
virtual  product  design  world  and  jointly  evaluate  design  issues,  ideas,  and  parameters  -- 
each  from  their  own  experience,  perspective,  viewpoint,  and  functional  responsibility. 


6 


The  functionalism  of  visualization  is  supported  by  Barkan  and  Iansiti  (1993)  and  Horton 
and  Radcliffe  (1995)  who  believe  that  a  VE  environment  would  offer  a  common  focus  for 
multidisciplinary  (managerial,  technical,  marketing,  and  manufacturing)  teams  to  resolve 
design  issues  by  sharpening  their  viewpoints  in  support  of  collaborative  design 
evaluation.  However,  there  is  a  need  to  empirically  evaluate  the  use  of  VE  technologies 
on  the  collaborative  design  review  process. 

In  summary,  there  is  a  need  to  investigate  the  value  of  various  commercial  visual 
display  systems  combined  with  cross-functional  team  collaboration  in  the  product  design 
review  environment.  The  reasons  are  that  this  marriage  has  potential  to  improve: 

1 .  Product  design  efficiency  by  decreasing  the  design  cycle  time, 

2.  Product  design  effectiveness  by  improving  design  quality,  and 

3.  Multi-functional  team  design  by  providing  a  mechanism  for  enhanced 
communication  and  better  decision  making. 

Statement  of  the  Problem 

The  application  of  emerging  VE  technologies,  to  the  product  design  and 
development  process  has  intrigued  many  researchers  and  practitioners  in  industry, 
government,  and  academia.  Using  VE  tools,  design  teams  can  assemble  to  evaluate  future 
designs  all  in  a  synthetic  environment  before  any  physical  prototypes  are  built.  This 
technology  creates  a  unique  capability  with  the  potendal  to  reduce  cost,  shrink  time  to 
market,  and  improve  overall  product  quality.  However,  a  proliferation  of  visual  display 
devices  in  the  commercial  market,  used  to  generate  3-D  stereoscopic  information,  has 
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provided  an  incentive  for  organizations  to  make  decisions  about  which  tools  to  use  to 
optimize  their  design  processes.  Today,  most  organizations  do  not  have  the  resources  to 
conduct  a  comparative  study  of  these  3-D  visual  display  devices  to  make  appropriate 
trade-off  determinations.  As  a  result,  their  decisions  are  often  arbitrary. 

The  purpose  of  this  study  was  to  evaluate,  through  comparative  empirical  testing 
and  data  analysis,  how  several  commercially-available  virtual  3-D  visual  display  systems 
contribute  to  cross-functional  team  collaboration  in  a  product  design  review 
environment.  Several  product  design  issues  that  need  to  be  addressed  are:  Which  visual 
display  devices  help  design  teams  detect  more  design  problems?  Which  help  design 
teams  find  these  design  problems  faster?  Which  visual  display  technologies  assist  in 
resolving  design  problems?  Do  these  tools  actually  assist  the  design  team  in 
accomplishing  improved  design  reviews,  thereby,  yielding  better  quality  solutions  and 
products?  Does  this  environment  improve  communication  and  reduce  the  number  of 
design  iterations  and  changes?  Is  the  overall  productivity  of  the  design  team  enhanced? 
Which  technologies  are  best  under  certain  circumstances? 

Definitions  are  presented  to  facilitate  reader  comprehension  of  words  and  phrases 
used  throughout  the  study.  Appendix  A  offers  a  list  of  abbreviations  and  acronyms. 

Definitions  of  Terms 

Asynchronous.  Mode  of  interaction  where  things  occur  at  different  times. 

Binocular  Omni-Orientation  Monitor  (BOOM).  A  viewing  device  mounted  to  a 
stand  that  allows  movement  with  6  degrees  of  freedom.  Using  mechanical  tracking,  the 
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stand  sends  position  and  orientation  information  to  the  computing  system,  and  the  user, 
who  holds  the  viewer  to  his  face,  can  use  it  to  view  a  3-D  environment.  (Stuart,  1996,  p. 
237) 

Cave  Automatic  Virtual  Environment  (CAVE).  A  projection  based  virtual 
environment  system  that  surrounds  the  user  with  four  screens.  Wearing  stereoscopic 
shutter  glasses  the  viewer  perceives  3-D  images. 

Cathode  Ray  Tube  (CRT).  The  most  common  display  technology  used  in 
computer  monitors.  The  technology  uses  a  sealed  glass  vacuum  tube  that  contains  an 
electron  beam  emitter,  or  gun,  a  deflection  yoke  that  focuses  the  beam,  and  a  phosphor- 
coated  screen  that  glows  for  a  fraction  of  a  second  when  struck  by  a  beam.  (Peddie,  1994, 
p.  230) 

Color  Monitor.  Color  monitors  use  three  primary  additive  colors:  red,  green,  and 
blue.  The  final  screen  color  on  the  video  screen  is  created  by  the  combination  of  these 
primaries.  The  strength  of  the  beam,  as  it  strikes  the  phosphor,  causes  it  to  illuminate 
with  brightness  to  match.  The  combination  of  all  three  phosphors  creates  color  in  a 
picture  element  (pixel).  (Peddie,  1994,  p.  231) 

Computer  Supported  Cooperative  Work  (CSCWT  Collaborative  work  between 
individuals  that  is  carried  out  with  the  support  of  communications  and  computing 
technology  (Stuart,  1996,  p.  237).  It  is  a  multi-disciplinary  field  of  study  that  performs 
research  in  the  area  of  automation  on  group  activities. 
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Computer  Aided  Desien  (CAD),  A  CAD  system  is  a  precision  drawing  tool  that 


speeds  up  the  design  process  by  automating  the  work  of  a  designer. 

Cvcle  Time.  The  interval  from  the  start  of  product  definition  until  that  product  is 
available  for  purchase  by  consumers.  (Sansone  &  Singer,  1992) 

Field  of  View  (FOV).  The  part  of  space  you  can  see  immediately,  without  moving 
your  head  or  your  eyes.  In  terms  of  a  visual  display,  the  FOV  is  the  visual  angle 
subtended  by  the  display  (Stuart,  1996,  p.  239). 

Haptic.  Refers  to  all  the  physical  sensors  that  generate  a  sense  of  touch  at  skin 
level  and  force  feedback  information  for  our  muscle  and  joints.  (Pimentel  &  Teixeira. 
1995,  p.  411) 

Head  Mounted  Display  (HMD).  A  headpiece  or  head-held  brace  with  viewing  or 
optical  devices  located  or  suspended  in  front  of  the  user’s  eyes.  These  optical  devices 
generate  images  seen  by  the  wearer  as  3-D.  (Larijani,  1994). 

Latency,  The  total  delay  time  between  a  user  action  and  the  system  responding  to 
this  action.  This  is  sometimes  referred  to  as  lag. 

Liquid  Crystal  Display  (LCD).  LCD  screens  are  made  up  of  liquid  crystal 
sandwiched  between  two  glass  plates.  They’re  typically  small  and  flat,  requiring  very 
little  power  for  operation.  (Peddie,  1994,  p.  240) 

Mental  Model.  A  model  that  a  user  forms  of  how  a  computer  system  or  program 
works.  It  can  be  conceived  as  the  users’  understanding  of  the  relationships  of  between  the 
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input  and  output  of  an  interface  design.  (Eberts,  1994,  pp.  140-141) 


Monoscopic.  Seeing  with  one  eye.  (Sekuler  &  Blake,  1994,  p.  513) 

Monoscopic  Display.  A  display  system  that  is  based  on  viewing  the  same  2-D 
image  with  both  eyes. 

Prototype.  A  variety  of  means  that  are  used  to  produce  preliminary  embodiments 
of  a  product,  and/or  its  subsystems.  (Barkan  &  Iansiti,  1993) 

Spatial  Resolution  (Monitors).  The  combination  of  the  number  of  vertical  and 
horizontal  lines  or  pixels  in  a  raster  display  device,  referring  to  how  sharply  an  image  can 
be  defined  on  the  screen  of  a  CRT.  Generally,  the  higher  the  numbers  are  equivalent  to 
the  finer  the  display. 

Stereoscopic.  Binocular  depth  perception  based  on  retinal  disparity.  (Sekuler  & 
Blake,  1994,  p.519) 

Stereoscopic  Display.  A  visual  display  that  presents  different  images  to  two  eyes 
in  order  to  produce  the  effect  of  stereopsis  to  the  viewer. 

Synchronous.  Mode  of  interaction  where  events  occur  at  the  same  time. 

Tactile.  The  sense  of  touch  or  pressure  applied  to  skin  (Pimentel  &  Teixeira, 

1995,  p.  416) 

Virtual  Environment  (VE).  A  high-end  user  interface  that  involves  real-time 
simulation  and  interactions  through  multiple  sensorial  channels.  These  sensorial 
modalities  are  visual,  auditory,  and  tactile.  (Burdea  &  Coiffet,  1994) 
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Virtual  Prototyping.  A  computer-based  simulation  of  systems  and  subsystems 
with  a  degree  of  functional  realism  comparable  to  a  physical  prototype  that  facilitates 

immersion  and  navigation.  Virtual  prototypes  are  used  for  test  and  evaluation  of  specific 
characteristics  of  candidate  design.  (Garcia,  Gocke,  &  Johnson,  1994,  p.  154) 

Virtual  Reality.  See  Virtual  Environments. 

Methodology 

Product  design  reviews  are  conducted  periodically  throughout  the  product  design 
and  development  process,  to  identify  problems,  formulate  solutions,  and  develop  a 
consensus  among  design  team  participants.  The  research  plan  of  this  study  emulates  the 
traditional  design  review  process  except  that  multi-functional  design  teams  identified  and 
solved  design  problems  with  the  assistance  of  four  VE  visual  display  technologies.  These 
technologies  are  a  helmet-mounted  display  (HMD),  binocular  omni  orientation  monitor 
(BOOM),  stereoscopic  glasses  and  monitor,  and  for  comparison  a  traditional  CAD-based 
computer  monitor. 

Participants  for  the  study  were  12  male  personnel  from  the  U.  S.  Army  Tank- 
automotive  Command  (TACOM).  Tests  were  conducted  at  TACOM’s  design  lab  in 
Warren,  Michigan.  Test  participants  were  grouped  into  three  person  design  teams  from 
the  following  functional  organizations:  new  equipment  training,  human  factors,  logistics 
and  maintenance,  configuration  management,  quality  assurance,  and  design.  Test  teams 
were  purposefully  balanced  with  similar  years  of  work  experience,  computer  knowledge. 
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education,  and  design  review  experience.  Participant  data  was  collected  using  several 
survey  instruments  and  independent  observation. 

An  exploratory  concept  vehicle  was  developed  as  the  test  model  for 
experimentation  and  data  collection.  This  concept  was  a  600-gallon  Tracked  Fuel  Trailer 
System  (IMS)  vehicle,  towed  behind  an  Abrams  Ml  A1  Main  Battle  Tank  to  provide  a 
continuous  fuel  supply  until  exhausted.  Four  I  M  S  subassemblies  were  used  as  the  focus 
of  the  design  review  task.  These  were  the  fuel  transfer,  fuel  container,  suspension,  and 
towing.  The  four  subassemblies  contained  design  problems  and  potential  design  issues 
related  to  mechanical,  ergonomic,  maintenance,  and  operational  functions. 

A  4x4  Graeco-Latin  Square  experimental  design  was  selected  for  its 
appropriateness  to  the  research  proposed  in  this  study.  The  treatments  that  were 
compared  were  the  four  visual  display  technologies.  The  three  blocking  factors  were  four 
design  teams,  four  different  TFTS  subassemblies,  and  four  experimental  orders. 

The  design  methodology  sought  to  provide  insight  into  how  commercial  visual 
display  systems  are  used  during  a  concept  design  review  (CDR)  process  and  what  value 
they  added.  Answers  to  the  following  questions  were  sought: 

1.  Error  Detection.  Do  design  teams  detect  and  identify  more  design  errors  when 
using  3-D  visualization  tools? 

2.  Time  to  Detect.  Do  design  teams  detect  and  identify  design  errors  more 
quickly  when  using  3-D  visualization  tools? 
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3.  Time  to  Resolve.  Do  design  teams  resolve  design  problems  more  quickly 
when  using  3-D  visualization  tools? 

4.  Perception  of  individual  visual  display  systems.  What  is  the  perceived  impact 
on  the  design  process,  the  quality  of  the  design  process,  and  the  physiological 
state  of  the  design  team  members  after  using  various  3-D  visualization  tools? 

5.  Preference  of  visual  display  systems.  What  are  preferences  of  usefulness, 
difficulty,  practicality,  stimulation  for  group  interactivity  and  development  of 
team  consensus  when  comparing  the  four  display  technologies? 

This  study  employed  several  human-computer  interaction  and  usability  evaluation 
techniques  to  collect  data  to  assess  the  research  questions.  These  approaches  were 
empirical  evaluation  of  the  task,  Likert-like  questionnaires,  human  interface  comparisons, 
and  open-ended  questions. 

Empirical  evaluation  of  the  task  captured  data  for  the  following  dependent 
measures:  (1)  the  number  of  design  errors  detected,  (2)  the  average  time  to  detect  a 
design  error,  and  (3)  the  average  time  to  resolve  a  detected  design  error.  An  independent 
test  observer  documented  design  team  performance  by  taking  continuous  time 
measurements  of  the  start  and  completion  times  for  each  problem  identified  and  solved 
by  the  design  team. 

A  VE  assessment  questionnaire  was  administered  to  each  test  participant  after 
completion  of  each  treatment  condition.  The  questionnaire  was  designed  to  assess  the 
strengths  and  weaknesses  of  the  technology-training  period,  the  design  review  process, 
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the  quality  of  the  design  review  process,  and  the  physiological  effects  of  exposure  to  the 
visual  display  systems. 

After  each  team  had  completed  experimentation  with  all  four  visual  display 
systems,  participants  were  asked  to  compare  the  technologies  by  ranking  them  from  the 
best  to  worst.  The  factors  assessed  were  the  usefulness  of  the  technology,  the  difficulty  of 
using  the  technology,  the  practicality  of  using  the  technology  for  CDRs,  how  helpful  the 
technology  was  in  stimulating  team  activity,  and  how  beneficial  the  technologies  were  for 
team  consensus. 

Limitations  of  Study 

The  following  limitations  can  be  applied  to  this  study: 

1.  The  study  was  limited  to  team  interaction  in  a  concept  design  review  environment. 

2.  A  single  conceptual  model  was  used. 

3.  The  study  was  restricted  by  the  technical  parameters  and  capabilities  of  the 
commercial  visual  display  devices  (equipment)  used  in  the  study. 

The  study  was  limited  to  the  performance  of  design  teams  during  a  CDR  task. 
Individual  participant  performance  was  not  addressed  in  this  study.  However,  perception 
and  preference  results  were  based  on  each  participant’s  responses  to  VE  assessment 
surveys. 

Another  study  limitation  was  that  the  design  teams  only  evaluated  a  single 
concept  model  for  the  study.  This  required  that  the  IM  S  model  be  separated  into  four 
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specific  subassemblies  and  that  test  participants  had  to  focus  on  only  one  subassembly  at 
a  time. 

Differences  between  the  technical  specifications  of  the  visual  display  systems 
(FOV,  resolution,  and  type  of  optics)  studied  were  also  not  the  focus  of  this  analysis.  The 
purpose  of  this  study  was  to  ascertain  the  value  of  typical  commercial  visual  display 
systems  on  the  collaborative,  CDR  process. 

Assumptions 

The  following  assumptions  were  made  concerning  the  conduct  of  the  study: 

1 .  Test  subjects  are  unbiased  towards  conventional  3-D  CAD  and  emerging  VE 
technologies,  and 

2.  The  knowledge  of  TACOM  test  subjects  was  representative  of  the  general  population 
within  the  area  of  military  vehicle  design  in  the  U.S.  Army. 

The  first  assumption  for  this  study  was  that  test  participants  were  not  biased 
towards  conventional  CAD  and  emerging  VE  technologies.  User  interface  evaluation 
techniques  often  incorporate  comparative  analysis  methods  between  status  quo  and 
alternative  interface  designs  (Nielsen,  1993).  It  is  assumed  that  test  subjects  are  open- 
minded  during  the  evaluation  of  new  technologies  and  how  they  are  used  in  a  product 
design  review  environment. 

The  second  assumption  presupposes. that  the  study’s  test  participants  represent  a 
cross-section  of  the  population  at  TACOM  and  others  involved  in  vehicle  design  reviews 
in  the  U.  S.  Army.  Study  participants  were  randomly  selected  from  several  functional 
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organizations  within  TACOM  for  assignment  to  design  teams.  Each  of  the  four  teams 
was  composed  of  one  designer  and  two  non-designers.  It  was  impractical  to  test  the 
entire  population  or  all  possible  combinations  of  functional  organizations  team 
membership.  The  sample  population  adequately  represented  the  total  population. 
Demographics  data  was  collected  and  assessed  to  verify  this  assumption  on  the 
representativeness  of  design  team  participants. 

Summary 

The  purpose  of  this  study  was  to  conduct  a  comparative  analysis  of  four 
commercial  3-D  visual  display  devices  and  to  determine  the  value  of  these  systems  to  a 
multi-functional  collaborative  team  design  environment.  It  has  been  hypothesized  that 
VE  visual  display  devices  can  offer  a  common  focus  for  multidisciplinary  teams  to 
resolve  design  issues  by  sharpening  their  viewpoints  in  support  of  collaborative  design 
evaluation  (Barkan  &  Iansiti,  1993;  Horton  &  Radcliffe,  1995).  Supporters  also  believe 
that  VEs  are  very  powerful  communication  tools  (Ellis,  1997).  When  these  technologies 
are  applied  to  product  design  it  is  hypothesized  that  these  systems  can  help  reduce  design 
cycle  time,  decrease  cost,  and  minimize  risk. 

The  methodology  for  this  study  was  a  4x4  Graeco-Latin  Square  design.  Testing 
was  conducted  at  an  U.  S.  Army  development  facility  in  Warren,  Michigan.  Participants 
were  assembled  into  three  person  design  teams  and  were  asked  to  conduct  a  design 
review  of  a  TFTS  concept  vehicle  system.  The  data  analysis  was  based  on  the 
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performance  of  12  test  participants.  Three  methods  were  used  for  data  collection: 

1 .  Measurement  of  test  team  performance, 

2.  Administration  of  questionnaires,  and 

3.  Observer  evaluation 

The  study  was  limited  to  the  performance  of  design  teams  during  a  CDR  task,  the 
use  of  a  single  model  for  evaluation,  and  to  the  technical  specifications  of  each  visual 
display  system  evaluated.  Assumptions  of  the  study  are  that  the  design  team  participants 
were  not  biased  towards  conventional  CAD  tools  or  emerging  VE  technologies  and  were 
representative  of  U.  S.  Army  vehicle  design  review  personnel. 

The  original  contribution  of  the  research  conducted  in  this  study  is  a  comparative 
analysis  of  four  commercial  visual  display  systems  used  during  a  design  review. 
Specifically,  the  study  assessed  the  value  and  differences  between  the  four  commercially 
available  visual  display  technologies.  Quantitative  data  collected  included  the  number  of 
errors  detected,  the  average  time  to  detect  a  design  error,  and  the  average  time  to  resolve 
detected  design  problems.  Supportive  qualitative  data  were  collected  assessing  the 
perceived  value  of  each  visual  display  system  and  preferences  between  them. 
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CHAPTER  2 


LITERATURE  REVIEW 

The  purpose  of  this  study  was  to  evaluate,  through  comparative  empirical  testing 
and  data  analysis,  how  several  commercially-available  virtual  3-D  visual  display  systems 
contribute  to  cross-functional  team  collaboration  in  a  product  design  review 
environment.  This  chapter  presents  the  results  of  a  literature  review  of  relevant  topics 
important  to  addressing  the  research  issues  and  identifying  design  process  improvements. 
These  topics  are  product  development,  the  concept  design  phase,  product  design  reviews, 
design  performance  measures,  the  collaborative  nature  of  design,  and  computer  assistance 
in  design— emerging  technologies. 

Current  product  development  methodologies  and  industrial  trends  are  important 
indicators  of  today’s  business  environment,  assisting  in  the  identification  of  more 
effective  and  efficient  product  development  processes,  a  critical  goal  of  today’s 
organizations. 

A  significant  amount  of  resources  are  expended  in  the  early  phases  of  the  product 
development  process,  in  early  concept  design.  For  this  reason,  concept  design  is 
investigated  in  terms  of  its  function,  how  visualization  plays  a  major  role,  and  how 
product  prototyping  influences  the  design  process. 
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Design  reviews  are  important  milestones  conducted  periodically  throughout  the 
product  design  and  development  process.  It  is  during  these  reviews  that  functional 
groups  collectively  evaluate  the  progress  and  determine  the  fate  of  the  product 
development  effort.  Design  review  methods,  advantages,  and  disadvantages  are 
discussed. 

It  is  also  important  to  know  when  a  product  has  reached  a  level  in  the  design  that 
provides  product  managers  with  more  surety  of  success.  This  is  critical  to  stability  in 
today’s  volatile  global  environment.  Definitions  of  design  performance  in  terms  of 
effectiveness  and  efficiency  are  reviewed. 

Today,  the  process  of  design  is  becoming  a  team-orientated  activity.  An  activity 
that  requires  teams  of  individuals  to  share  information,  discuss  options,  formulate 
consensus  and  execute  decisions.  Therefore,  concepts  of  collaborative  work  in  product 
design  are  explored.  Two  bodies  of  knowledge  are  reviewed:  concurrent  engineering  and 
CSCW.  The  first  offers  insight  into  the  impact  of  organizational  teaming  on  the  product 
design  process  and  the  later  offers  insight  into  the  automation  of  group  work. 

Overtime,  the  product  design  process  has  evolved  as  it  has  been  integrated  with 
computer  and  communication  technologies.  One  emerging  technology  are  virtual 
environments,  they  have  the  potential  to  impact  the  design  process  for  they  offer  an 
approach  which  maximizes  the  human  visualization  capability  and  allows  multifunctional 
teams  the  ability  to  simultaneously  interact  in  the  same  design  environment.  A 
description  of  VE  systems  is  provided.  Also,  the  value  of  VE  immersion,  3-D  display 
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research,  current  multi-user  systems  and  applications  to  collaborative  virtual  design  were 
investigated. 

Product  Development 

The  product  development  process  has  been  characterized  as  a  systematic  method 
of  evolving  a  product  from  idea  conception  to  product  release  to  customers.  Pugh  (1991 ) 
defines  product  development  as  a  systematic  activity  necessary  to  produce  a  ‘total 
design,’  from  the  identification  of  the  market/user  need,  to  selling  of  the  successful 
product,  process,  people,  and  organization.  Also  identified  in  his  total  design  activity 
model,  is  an  iterative  process  based  on  a  progression  through  specific  core  activities: 
market,  specification,  concept  design,  detail  design,  manufacture,  and  sales.  He 
emphasizes  the  iterative  nature  and  flexibility  required  in  design. 

Ulrich  and  Eppinger  (1995)  identify  a  structured  methodology  for  the  product 
development  process.  The  methodology  begins  with  concept  development,  and 
progresses  through  system  level  design,  detail  design,  testing  and  refinement  and 
production  ramp-up.  These  researchers  have  identified  five  specific  dimensions  to  assess 
the  performance  and  success  of  a  product  development  effort:  product  quality,  product 
cost,  development  time,  development  cost,  and  development  capability.  Their  belief  is 
that  product  development  is  an  interdisciplinary  activity  involving  marketing, 
engineering,  design,  and  manufacturing. 
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Modem  society  is  reliant  on  the  fast  paced  movement  of  technology.  This  rapid 
growth  has  forced  manufacturers  to  produce  products  that  incorporate  these  ever- 
evolving  technologies.  Customer  expectations,  more  than  ever,  are  forcing  developers  to 
produce  products  that  have  shorter  life  cycles.  Ohmae  (1995)  notes  that  the  model  life 
for  audio  components  is  about  6  months  and  for  early  facsimile  machines  a  mere  4 
months.  To  remain  competitive  it  has  become  essential  for  producers  to  keep  pace  with 
technology  changes  and  live  with  shorter  product  development  cycles. 

Emphasis  has  been  placed  on  speeding  up  the  time  from  product  idea  or 
conception  to  product  launch  because  being  first  offers  a  great  competitive  advantage. 
Vesey  (1991),  in  a  study  of  high-technology  products,  reported  that  if  the  products  were 
six  months  late  in  entering  the  market  they  earned  33  percent  less  over  a  five  year  period 
than  they  would  have  earned  had  they  been  introduced  on  time.  A  conclusion  reached  is 
that  the  resultant  potential  for  market  loss  makes  timeliness  of  product  introduction  a 
critical  success  factor. 

Cordero  (1991)  surveyed  several  techniques  for  reducing  product  development 
times.  He  indicates  that  by  better  managing  product  development  speed,  an  organization 
can  achieve  the  following  benefits:  a  faster  response  to  market  needs,  reduced  product 
cost,  and  increased  product  quality.  He  categorized  four  distinct  management  strategies: 
organize  product  development,  organize  product  manufacturing,  use  miscellaneous 
techniques,  and  use  computer-aided  techniques.  Of  particular  importance  to  this  study 
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are  two  of  these  strategies  :  organizing  product  development  and  use  of  computer  aided 
techniques. 

Organizing  the  product  development  structure  can  be  further  broken  into  four 
approaches  --  some  better  than  others.  The  first,  a  traditional  phased  approach  in  which 
functional  representatives  are  gathered  into  project  teams.  In  these  teams,  representatives 
may  work  autonomously  from  each  other  and  frequently  at  different  times  and  locations. 
Cordero  (1991)  identified  several  disadvantages  of  this  approach.  They  are:  much  time  is 
wasted  when  physically  separate  functional  representatives  need  to  communicate,  time  is 
also  lost  when  there  is  a  need  to  conform  to  policies  and  rules,  and  decisions  have  to  go 
through  several  layers  of  functional  and  project  approval.  The  second,  a  faster  phased 
approach,  achieves  acceleration  by  integrating  functional  members  and  bringing  them 
physically  closer  together  thus  encouraging  open,  cooperative  communication.  With  this 
approach,  control  is  normally  delegated  to  project  teams.  The  third,  a  concurrent 
approach,  occurs  when  functional  representatives  are  closely  integrated  into  project  teams 
and  make  decisions  based  on  shared  information  about  market  needs,  technical 
feasibility,  and  product  costs.  In  this  way  the  team  ensures  that  product  development 
simultaneously  considers  all  interfunctional  requirements.  This  concurrent  approach 
saves  time  and  conflict  downstream  by  spending  time  up-front,  however  this  approach 
introduces  risk  by  reducing  managerial  control.  The  fourth,  a  contingency  approach,  is 
dependent  on  the  type  of  product  being  produced.  Cordero  (1991)  postulates  that  this 
latter  faster  phased  approach  appears  more  appropriate  for  minor  product  changes  and  the 
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concurrent  approach  is  more  appropriate  for  cases  of  incremental  product  innovation  and 
multiple  product  solutions. 

Cordero  (1991)  also  indicates  that  computer-aided  techniques  can  help  speed  a 
company’s  product  development  strategy.  These  techniques  include  computer-aided 
design  (CAD),  computer-aided  engineering  (CAE),  computer-aided  process  planning 
(CAPP),  and  computer-aided  manufacturing  (CAM).  In  his  research,  Cordero  found  that 
computer-aided  techniques  facilitate  functional  cooperation  because  they  increase  the  rate 
of  information  dispersion  and  that  they  can  also  improve  quality  and  reduce  cost. 

Eisenhardt  and  Tabrizi  (1995)  studied  data  from  72  new  product  development 
projects  and  concluded  that  accelerated  product  development  can  be  attributed  to  a 
strategy  that  relies  on  improvisation,  experience,  flexibility,  multiple  design  iterations, 
extensive  testing,  frequent  milestones,  a  powerful  team  leader,  and  a  multi-functional 
team.  Interestingly,  they  also  found  that  planning  and  rewarding  for  schedule  attainment 
are  ineffective  means  of  accelerating  the  pace  of  product  development. 

Brown  and  Eisenhardt  (1995)  conducted  a  product  development  literature  survey 
and  identified  three  approaches:  product  development  as  a  rational  plan  (financial 
performance  of  a  product),  communication  web  (effects  of  communication  on 
performance),  and  disciplined  problem  solving  (effect  of  the  product,  team  and  suppliers 
on  the  process).  These  investigators  concluded  that  all  three  demonstrated  that  effective 
group  processes  increase  the  flow  of  information  and  are  essential  for  high  performing 
product  development  processes.  Their  research  concluded  that  for  a  stable,  mature 
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product  use  of  extensive  planning  and  overlapped  development  stages  are  appropriate.  In 
contrast,  when  there  is  uncertainty  in  the  design  process,  more  experiential  tactics  are 
more  productive,  including  frequent  iterations  of  product  design,  extensive  testing,  and 
short  frequent  milestones. 

Throughout  the  product  development  literature  review  one-theme  surfaces  --  one 
which  guides  successful  product  developers  in  achieving  a  goal  of  decreasing  the  time  to 
market.  The  theme  is  that  the  use  of  multi-functional  and  empowered  teams,  frequent 
design  iterations,  extensive  testing,  short  product  milestones,  and  flexible  leadership  has 
proved  to  be  successful  predictors  in  the  attainment  of  a  minimized  product  development 
cycle.  It  can  also  be  reasoned  that  streamlining  any  phase  of  product  development  will 
improve  the  overall  process.  In  particular,  the  product  concept  design  phase  is  a 
significant  contributor  to  final  product  cost,  form,  reliability,  and  market  acceptance. 

Any  strategy  or  methodology  that  offers  potential  to  decrease  or  improve  concept  design 
time  merits  further  investigation. 

Concept  Development  and  Product  Design 

The  time  to  market  and  product  cost  are  dominated  by  decisions  made  in  the  early 
stages  of  the  design  process.  Will  (1991)  reiterates  results  identified  in  a  National  Design 
Strategy  report  that  indicated  that  85%  of  product  development  costs  are  determined 
before  the  product  design  is  released  to  manufacturing.  In  a  related  study,  similar 
findings  were  shown.  Garcia,  Gocke,  and  Johnson  (1994)  indicate  that  by  the  time  10% 
of  total  funds  of  a  project  are  spent,  approximately  90%  of  a  product’s  development  costs 
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are  established.  In  other  words,  the  most  cost-determining  element  in  the  product 
development  effort  occurs  early  in  the  product  development  cycle  as  planning  and  design 
decisions  are  made.  These  decisions  are  attempts  to  identify  and  reduce  risk  and  improve 
product  development  efficiency. 

Concept  Generation 

Ulrich  and  Eppinger  (1995)  describe  the  concept  development  phase  of  the 
product  development  process.  They  believe  that  the  process  begins  with  initiated 
customer  requirements  that  are  translated  into  functional  specifications.  These  functional 
specifications  are  then  used  to  generate  product  concepts.  Selection  methods  are  then 
applied  to  the  product  concepts  to  systematically  select  the  concept  that  best  meets 
customer’s  needs.  Concept  generation  involves  the  actual  creation  and  development  of 
product  alternatives.  It  embraces  conceptual  thinking,  requiring  both  the  logical  and 
creative  sides  of  the  human  brain  (Goel  &  Pirolli,  1992).  To  develop  a  more  efficient, 
effective  concept  design  process  one  must  understand  the  human  aspects  of  the  concept 
design  process  itself. 

Mitchell  (1994)  describes  three  characteristics  of  design:  a  problem  solving 
activity,  a  knowledge-based  activity,  and  social  activity.  Mostow  (1985)  further 
categorized  design  as  a  special  kind  of  problem  solving  activity,  and  a  complex  problem¬ 
solving  event.  In  a  process  where  the  parameters  and  goals  are  not  easily  defined  and  the 
variables  are  often  qualitative  rather  than  quantitative,  complex  interrelationships 
between  variables  exist  (Mostow,  1985;  Derrington,  1987).  The  conceptual  generation 
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process  involves  a  mental  formulation  of  future  state  of  affairs  where  the  products  of  the 
design  are  external  representations  of  such  possible  states  (Goel  &  Pirolli,  1992).  These 
investigators  also  believe  that  the  design  problem  solving  activity  involves  the  following 
sequence  of  steps: 

1 .  An  exploration  and  decomposition  of  the  problem, 

2.  An  identification  of  the  interconnections  among  the  components, 

3.  The  solution  of  the  subproblems  in  isolation,  and 

4.  The  combination  of  the  partial  solutions  into  the  problem  solution  (p.  397) 

This  is  a  systematic,  logical  approach  where  incremental  development  of  design 
ideas  are  generated,  retained,  messaged  and  developed  until  they  reach  their  final  form. 
All  associated  knowledge  is  brought  into  the  design  space. 

Design  as  a  knowledge-based  activity,  implies  that  good  human  designers  are 
extremely  knowledgeable  about  the  domains  in  which  they  operate.  Mitchell  (1994) 
discusses  that  to  be  able  to  creatively  develop  new  solutions  and  designs  a  good 
knowledge  base  of  product  characteristics,  materials,  relationships,  and  past  product 
development  experiences  needs  to  be  present. 

The  process  of  design  is  becoming  more  reliant  on  team  structures  and  dynamics, 
causing  the  process  to  be  more  of  a  social  activity  because  there  is  a  need  to  maintain  a 
shared  understanding  amongst  all  participants  in  the  design  process  (Holt  &  Radcliffe, 
1991).  These  multi-functional  teams  proceed  by  exchanging  proposals,  arguments,  ideas. 
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and  seek  to  form  a  consensus.  Knowledge  is  transformed  into  a  common  pool  of 
information  where  interaction  and  conflict  resolution  can  take  place  (Mitchell,  1994). 

Concept  Selection 

Concept  selection  is  part  of  the  design  process  where  various  product  concepts  are 
analyzed  and  sequentially  eliminated  to  identify  one  preferred  concept.  Ulrich  and 
Eppinger  (1995)  believe  that: 

Concept  selection  is  a  process  of  evaluating  concepts  with  respect  to 
customer  needs  and  other  criteria,  comparing  the  relative  strengths  and 
weaknesses  of  the  concept,  and  selecting  one  or  more  concepts  for  further 
investigation  or  development,  (pp.  106-107) 

Concept  selection  is  a  critical  phase  in  the  product  development  process  because  it 
is  at  this  stage  that  important  decisions  are  made  regarding  the  design  and  eventual 
production  of  the  product.  Kmetovicz  (1992)  describes  product  alternative  analysis  as  a 
phase  where  ideas  that  were  synthesized  and  prototyped  are  subject  to  evaluation  and 
analysis  by  a  cross-functional  team.  It  is  at  this  point  that  decisions  that  help  shape  the 
future  are  produced.  However,  evaluating  teams  must  ensure  that  promising  concepts  are 
not  eliminated  prematurely,  thereby  missing  market  opportunity  and  that  poor  choices  are 
not  made,  which  could  lead  to  potential  failure.  Decisions  that  are  made  in  the  earliest 
stages  of  the  design  process  will  impact  the  entire  development  effort.  Spending  time  up¬ 
front  where  mistakes  are  cheaper  and  easily  fixed  is  crucial  to  developing  products  within 
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budget  and  schedule.  These  activities  are  often  referred  to  as  a  divergent-convergent 
process.  Divergent  because  within  the  selection  process  a  set  of  concepts  under 
consideration  is  narrowed.  Convergent  because  evaluations  may  produce  additional 
concepts  as  concepts  are  combined  or  improved. 

Kmetovicz  (1992)  describes  four  decision-making  approaches  used  during  the 
conceptual,  analysis  stage  of  product  development.  These  are  odds-based,  intuitive, 
analytical,  and  a  combination  of  all  three  approaches.  An  odds-based  style  is  based  on 
the  probability  of  being  right.  The  researcher  believes  that  this  decision  making  approach 
should  be  used  when  the  quantity  and  quality  of  information  is  low  and  the  error 
tolerance  is  high.  He  further  explains  that  the  intuitive  style  should  be  used  when  the 
information  is  limited  and  the  decision-maker  is  a  good  reader  of  the  environment  and 
possesses  good  information.  These  decisions  are  made  based  on  instinct  and  feeling.  In 
the  third  type,  using  the  analytical  decision  style,  the  decision-maker  relies  on  accurate 
information.  Here  the  information  needs  to  be  of  high  quality  and  quantity.  Kmetovicz 
(1992)  believes  that  analytical  methods  should  be  used  when  the  decision  has  little  or  no 
margin  for  error.  The  last  approach  involves  a  combination  of  all  three  methods.  The 
researcher  implies  that  there  is  no  specific  approach  and  that  sometimes  decision-makers 
must  consider  the  benefits  from  all  the  methods. 

Nippani  (1994)  offers  a  thorough  evaluation  of  several  existing  analytical 
techniques  that  have  been  proposed  for  the  evaluation  and  selection  of  conceptual  'oduct 
designs.  These  are:  weighting  techniques,  Pugh  concept  selection,  two-stage 
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methodology,  electre  methods,  fuzzy  methodologies,  utility  theory,  and  Analytical 
Hierarchy  Process.  The  advantage  of  using  these  approaches  is  that  they  are  structured, 
theory-based  methodologies.  Some  of  these  techniques  force  product  development  teams 
to  interact  and  develop  evaluation  criteria  in  the  earliest  stages  thus  propagating  team 
synergy  yielding  improved  concept  selections. 

Mental  Imagery,  Visualization,  and  Design 

Mental  images,  physical  images  and  virtual  images  all  play  a  role  in  aiding  the 
human  in  the  product  design  process.  First,  the  relationship  between  mental  imagery, 
visualization,  and  design  will  be  examined  in  a  historical  perspective.  Secondly,  an 
examination  of  physical  and  virtual  product  prototyping  methods  will  be  presented. 

Liston  and  Stanley  (1964)  summarized  effective  methods  for  conceiving, 
describing,  proving  and  communicating  new  product  ideas.  Creative  engineering,  a  term 
coined  by  the  researchers,  means  the  part  of  technology  which  uses  discoveries  of 
scientific  research  to  fashion  ideas  for  products  useful  to  man.  They  also  identify 
creative  product  design  as  a  non-routine  processes  that  is  a  mental  process  that  is  directed 
at  envisioning  or  imagining  solutions  to  problems.  Designers  often  describe  the  way  they 
deal  with  engineering  design  problems,  by  their  use  of  these  mental  images.  Mental 
images  are  analog  representations  that  permit  information  to  be  processed  in  a  holistic, 
parallel  way.  Mental  images  include  a  great  amount  of  information  about  the  objects 
being  presented,  which  can  be  used  when  solving  design  problems. 
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Ideas  or  solutions  to  a  problem  must  be  translated  into  a  picture  or  as  referred  to 
in  the  current  literature  as  a  mental  model.  Alabastro,  Beckmann,  Gifford,  Massey,  and 
Wallace  (1995)  maintain  that  humans  constantly  develop  mental  models  of  the  world 
around  them,  and  that  these  models  are  internal  mental  replicas  which  have  the  same 
relation-structure  as  the  objects  they  represent.  As  more  detail  is  accumulated  in  the 
mental  model  it  becomes  increasingly  more  difficult  to  keep  all  the  information  in  sharp 
focus.  There  is  a  limit  to  the  size  and  complexity  of  mental  images  that  can  be  sharply 
focused  in  the  conscious  mind 

External  assistance  in  the  form  of  pictorial  sketches  assists  the  designer  by 
providing  visual  stimulus  and  offers  a  feedback  mechanism  which  assists  the  conscious 
mind  in  keeping  details  in  focus  thus  stimulating  the  subconscious  mind  to  further 
complete  the  mental  model.  Ullman,  Wood,  and  Craig  (1989)  studied  the  effects  of 
drawing  and  sketching  on  the  mechanical  design  process.  Five  mechanical  design 
engineers  of  varying  experience  were  given  initial  specifications  for  a  simple  industrial 
design  problem.  The  engineers  were  asked  to  solve  a  problem.  During  the  experiment 
their  verbal  reports,  drawings,  and  gestures  were  video  and  audiotaped  for  6-10  hours. 
Analysis  of  the  data  concluded  that  86%  of  the  design  actions  was  in  sketching,  drawing, 
or  drafting.  According  to  the  researchers,  it  appears  that  drawing  was  done  by  engineers 
to  aid  their  cognitive  ability  to  visualize  and  solve  the  problem. 

The  field  of  human  computer  interaction  has  embraced  the  concept  of  mental 
models  as  it  offers  insight  into  computer  system  design.  Eberts  (1994)  discusses  the  use 
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of  mental  models  for  the  development  of  computer-user  interface  designs.  A  mental 
model  is  a  representation  that  users  form  of  how  a  computer  system  works.  This  model 
assists  the  user  in  understanding  the  current  state  of  a  system  and  provides  a  context  for 
predicting  future  events. 

Eisentraut  (1995)  identified  the  importance  of  mental  images  in  the  design 
process  in  the  following  way:  the  pictorial  knowledge  base  serves  as  a  source  for 
searching  possible  solutions,  design  solutions  can  be  enlarged,  and  the  images  can 
stimulate  the  evaluation  of  alternative  solutions.  A  conclusion  reached  is  that  a  limitation 
of  mental  imagery  is  the  capacity  of  human  memory.  Sketching  and  drawing  play  an 
important  role  in  relieving  this  human  capacity  limitation.  The  use  of  physical  or  virtual 
prototypes  can  add  more  clarity  for  filling  in  the  details  of  ideas  for  complex  products. 

Physical  and  Virtual  Prototyping 

Lanz  (1985)  defines  prototyping  as  a  functional  form  of  a  new  type  of  design. 

The  author  indicates  that  creation  of  a  prototype:  pleases  users,  reduces  development 
costs,  decreases  communication  problems,  slashes  calendar  time,  produces  the  right 
system  the  first  time,  and  cuts  manpower  needs.  In  the  past,  product  manufacturers 
created  physical  prototypes  and  mockups  so  people  could  see,  touch,  and  generally 
experience  a  product  (Larson-Mogal,  1994).  Products  evolved  from  paper-based 
descriptions  of  product  concepts,  to  partial  prototypes,  to  full  prototypes,  and  eventually 
to  a  complete  product.  Cost  and  risk  are  dependent  on  the  level  of  testing  with  a 
complete  product  being  at  the  high  end. 
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Horton  and  Radcliffe  (1995)  have  investigated  a  current  understanding  of  the  role 
of  prototyping  during  the  design  process.  These  investigators  created  a  four- faceted  view 
of  the  prototyping  effort:  problem  solving  (solving  problems  by  testing  a  solution), 
exploration  and  serendipity  (uncover  unexpected  problems  and  questions),  process  (build 
a  better  understanding  of  the  design),  and  sharing  (design  participant  can  converge, 
express,  negotiate).  From  their  perspective,  the  physical  aspects  of  prototypes  help 
materialize  a  design,  and  offer  a  common  focus  for  multi-disciplinary  groups  to  resolve 
design  issues  by  sharpening  their  viewpoints. 

Prototyping  is  an  integral  part  of  the  product  design  process,  and  has  become  a 
key  enabling  technology  in  reducing  the  time  to  market  for  new  products  by  identifying 
design  flaws  before  tooling  and  manufacture  begins.  According  to  Barkan  and  Iansiti 
(1993),  the  physical  nature  of  prototypes  offers  two  benefits:  they  provide  designers  and 
design  teams  with  a  mechanism  for  product  visualization,  and  function  as  a  real  platform 
for  collaborative  design  evaluations.  Managerial,  technical,  marketing,  and 
manufacturing  can  also  derive  benefits  from  prototypes  because  they  can  serve  as  a  basis 
for  rapid  learning,  team  unification,  and  consensus  building. 

Barkan  and  Iansiti  (1993)  also  indicate  that  all  forms  of  prototypes  are  a  powerful 
means  of  resolving  crucial  questions  quickly,  can  help  to  provide  a  common 
understanding  of  the  product  which  they  model,  and  serve  as  an  integrating  force  for  all 
members  of  a  multi-functional  organization.  Early  prototyping  reduces  the  risk 
associated  with  innovation  because  problems  are  detected  and  corrected  early. 
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Prototyping  can  also  contribute  to  a  reduction  in  product  development  time  because 
problems  can  be  reconciled  earlier  in  more  flexible  stages  of  design  --  at  a  point  when 
there  are  less  cost  and  difficulties  associated  with  design  changes. 

However,  the  benefits  of  physical  prototyping  can  be  overshadowed  by  their 
expensive,  repetitive  nature.  Physical  prototypes  frequently  require  manual  tooling, 
skilled  assembly,  delicate  testing  instrumentadon,  and  excessive  time  spent  interpreting 
test  data.  The  prototyping  process  can  become  a  revolving  sequence  of  events: 
identifying  problems,  applying  lessons  learned,  revising  physical  prototypes,  and  finally 
performing  re-evaluation  of  the  entire  prototype.  All  of  steps  are  time  consuming. 

Application  of  advanced  computer  simulation  technology  in  the  prototyping 
process  has  resulted  in  the  development  of  a  concept  called  virtual  prototyping.  Garcia. 
Gocke,  and  Johnson  (1994)  have  assessed  the  feasibility  of  using  virtual  prototypes  in 
the  Department  of  Defense  (DOD)  acquisition  process.  They  define  virtual  prototyping 
as: 

A  computer  based  simulation  of  systems  and  subsystems,  which  exhibits  both 
geometric  and  functional  realism.  This  three-dimensional  virtual  mock-up  may  be 
used  to  evaluate  prototypes  or  concepts,  and  provides  a  common  platform  from 
which  all  functional  disciplines  (design,  test,  manufacturing,  logistics,  training, 
and  operations)  can  work.  (Garcia,  Gocke,  &  Johnson,  1994,  p.  1-8). 

Lee  (1995)  describes  virtual  prototyping  as  visualizing  and  testing  computer- 
aided  design  models  on  a  computer  before  they  are  physically  created. 
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Virtual  prototypes  can  dramatically  reduce  the  need  for  expensive,  time 
consuming  physical  mockups  (Gottschalk  &  Machlis,  1994).  Burdea  and  Coiffet  (1994) 
estimate  that  the  time  from  concept  to  production  could  be  shortened  by  approximately 
50%  by  eliminating  the  hardware  iterative  test-build  cycles  required  in  the  traditional 
prototyping  process.  Ellis  (1997)  observes  that  when  General  Motors  uses  digital 
prototypes,  the  needed  design  changes  that  would  normally  take  40  to  50  weeks  before 
production  begins  are  being  discovered  100  weeks  earlier.  The  investigator  estimated 
that  savings  of  $80  million  dollars  were  realized  after  making  a  $5  million  dollar 
investment  in  the  virtual  prototype.  This  approach  is  becoming  an  increasingly  popular 
way  to  refine  design  assumptions  and  to  improve  new  products  (Lee,  1995).  Today,  the 
DOD  is  relying  on  the  use  of  virtual  prototypes  to  solve  the  problem  of  shrinking 
budgets,  diverse  requirements,  and  quick  changes  in  operation  (Ashley,  1995;  Hamit, 
1995;  Garcia  et  al.,  1994;  Kitfield,  1994).  In  comparison,  industry  is  facing  similar 
challenges  and  many  companies  are  investing  in  computational  methods  of  improving  the 
design  process  (Hedberg,  1994;  Keebler,  1993). 

The  development  of  mental  models  through  spatial  visualization  of  new  products 
through  the  use  of  physical  prototypes,  pictorial  images,  or  virtual  design  replicas  is 
evident  in  processes  of  design.  By  their  nature,  prototyping  efforts  are  conducive  to 
design  review  activities  because  they  offer  a  platform  and  mechanism  for  evaluation  of 
future  design  concepts.  For  this  reason,  product  design  review  activities  are  explored. 
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Product  Design  Reviews 


Product  design  reviews  have  been  described  in  varying  ways.  Juran  and  Gryna 
(1993)  define  these  reviews  as  a  formal,  documented,  comprehensive,  and  systematic 
examination  of  a  product  design  to:  evaluate  the  design  requirements  and  the  capability 
of  the  design  to  meet  these  requirements,  and  to  identify  problems  and  propose  solutions. 
Pugh  (1990)  describes  the  design  review,  which  he  also  calls  a  design  audit,  as  an 
essential  part  of  modem  industrial  practice.  These  reviews  provide  a  mechanism 
whereby  the  total  design  activity  can  be  carried  out  in  a  balanced  manner,  leading  to 
improved  designs  and  products. 

Fox  (1993),  Juran  and  Gryna  (1993),  and  Pugh  (1990)  all  suggest  that  design 
reviews  be  conducted  at  several  phases  of  the  progression  of  the  design:  minimally  at  the 
conceptual  stage,  detail  design  stage,  prototyping  stage,  final  design  stage,  and  before 
production  begins.  At  each  review,  comparisons  and  revisions  should  be  made  using  up- 
to-date  information  on  market  shifts,  reliability,  maintainability,  producibility, 
appearance,  cost,  and  enhanced  knowledge  of  product’s  design. 

The  formal  review  process  helps  to  manage  all  aspects  of  the  design  process. 
Hales  (1993)  supports  the  claims  that  regular  design  reviews  help  monitor  work  in 
progress  for  any  product  development  effort.  Fox  (1993)  further  iterates  that  information 
and  understanding  gained  from  the  reviews  are  used  to  make  decision  on  whether  to 
foreword  to  the  next  phase  of  the  development  process  or  whether  to  revise  the  program 
strategy. 
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Another  important  objective  of  design  reviews  is  to  integrate  knowledge  from 
various  functional  areas  into  one  common  pool  of  knowledge  through  consensus 
development.  To  make  this  possible,  people  who  contribute  to  the  design  review  must 
represent  several  different,  but  relevant  functional  organizations.  These  functional 
members  are  best  suited  to  identify  the  strengths  and  weaknesses  of  the  product 
development  effort  because  in  some  way  the  product  and  its  design  affect  them.  The 
combined  source  of  information  these  representatives  possess  promotes  the  achievement 
of  an  optimal  design  by  creating  a  unique  place  where  information  exchange,  interaction, 
and  conflict  resolution  can  take  place  (Mitchell,  1994).  In  this  way,  all  of  the  people  who 
have  a  part  to  play  in  making  the  design  and  product  successful  are  involved. 

Team  member  interaction  plays  a  major  role  in  the  product  design  review  process. 
Teams  have  the  capacity  to  plan,  organize,  make  decisions,  communicate,  and  negotiate 
the  actions  to  achieve  the  teams'  objectives  (Floyd  &  Turner,  1989).  In  collaborative 
work  environments,  there  is  a  need  to  communicate  design  decisions  and  to  coordinate 
the  creative  process  among  diverse  disciplines.  Inter-team  member  communications  is 
required  to  provide  information  sharing,  resource  allocation,  problem  solving,  and 
negotiation. 

Advantages  of  Team  Product  Design  Review 

Wilson  (1993)  discusses  several  advantages  of  team-based  problem  solving. 
Teams  provide  better  solutions  and  these  solutions  are  usually  more  cost  effective.  Team 
results  and  solutions  become  the  ownership  of  all  members.  This  collective  problem 
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solving  helps  to  ensure  that  various  functions  within  an  organization  buy  into  the  product 
and  the  design  approach,  because  representative  participants  have  a  stake  in  the  outcome 
of  the  project.  Teams  generally  require  less  management  control,  since  critical  decisions 
are  placed  in  the  hands  of  experts  instead  of  traditional  management.  Teams  also  provide 
a  synergy  of  expertise. 

Some  reasons  for  the  usefulness  of  design  reviews  are  as  follows:  they  give  the 
team  official  time  to  look  at  their  progress  from  a  fresh  viewpoint,  they  provide 
documented  status,  and  they  can  achieve  their  main  goal  to  confirm  that  the  program  is 
on  the  right  track  or  is  being  corrected  (Fox,  1993). 

Group  problem  solving  increases  understanding  and  commitment,  participation 
increases  the  likelihood  of  good  solutions  and  of  their  effective  implementation  (Fox, 
1987).  Participation  helps  us  to  know  the  why  as  well  as  the  what  that  are  involved,  and 
gives  individuals  a  stake  in  what  happens.  Other  benefits  of  participation  are  enhanced 
team  spirit,  increased  respect  for  the  team  leader,  and  increased  self-respect. 

People  also  want  to  participate  because  group  members  want  a  role  in  defining, 
analyzing,  and  solving  problems  that  concern  them  in  their  community  and  workplace. 

On  an  average,  several  people  will  produce  more  and  better  solutions  to  a  nonroutine 
problem  than  will  a  single  person.  Participation  in  goal  setting  and  problem  solving 
increases  an  individual’s  understanding  of  what  is  to  be  done  and  of  how  it  will  be  done. 
Participation  in  group  problem  solving  is  one  of  the  most  effective  means  for  gaining 
commitment.  It  is  better  to  have  created  solutions  to  problems  based  on  team  consensus 
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and  team  input.  Multi-functional  team  interaction  and  sharing  of  information  gives  teams 
the  capability  to  make  more  informed  and  better  decisions. 

Disadvantages  of  Team  Product  Design  Review 
Cross-functional  project  teams  do  not  always  guarantee  effective  product 
development  (Clark  &  Fujimoto,  1991),  and  the  use  of  groups  in  product  design  can 
cause  problems.  The  disadvantages  of  group  problem  solving  are  concerned  mainly  with 
the  requirement  for  effective  communications.  If  communication  channels  are  broken, 
then  group  synergy  is  destroyed.  Multi-functional  teamwork  involves  getting  the  right 
data  to  the  right  place  at  the  right  time  and  in  the  right  format.  Communication  is  all 
about  how  an  organization  distributes  and  disseminates  critical  information. 

Today,  a  design  team  may  be  in  one  location  or  geographically  dispersed 
(Hedberg,  1994),  adding  an  extra  burden  to  the  design  process  because  communication 
may  become  a  major  issue.  Exchanging  viewpoints  and  ideas  while  sharing  product 
information  is  critical  to  making  more  informed  decisions  regarding  a  product’s  design  in 
a  complex  environment  that  incorporates  multiple  person  interactions.  Frequent  meetings 
are  necessary  to  keep  communications  flowing. 

Regular  meetings  can  help  the  review  process  and  provoke  important  decisions. 
However,  for  those  in  distant  locations  travel  time  and  associated  expenses  are  costly 
(Yager,  1993).  These  expenses  and  disruptions  to  normal  work  can  inhibit  the  regularity 
of  meetings  and  the  spontaneity  of  team  interactions,  and  act  as  a  barrier  to  successful 
implementation  of  team  interaction  techniques  (Maxfield,  Fernando,  &  Dew,  1995). 
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What  is  necessary  to  overcome  these  problems  is  the  development  of  better  and  more 
effective  processes,  which  promote  communications  and  sharing  of  information. 

Better  design  team  performance  can  be  translated  into  improved  processes, 
reduced  cost,  and  increased  product  profit.  The  next  section  describes  measures  of  design 
performance  in  terms  of  effectiveness  and  efficiency  of  the  design  process. 

Measures  of  Design  Performance 

Hales  (1993)  describes  how  a  design  team  is  expected  to  be  efficient  (doing  things 
right)  and  effective  (doing  the  right  things).  Soukhanov  (1984)  defines  efficiency  as  the 
degree  to  which  quality  is  exercised  or  as  the  ratio  of  the  useful  output  to  the  total  input 
of  the  system.  This  researcher  defines  effectiveness  as  producing  or  designed  to  produce 
a  desired  effect. 

Clark  and  Fujimoto  (1991)  believe  that  three  outcomes  of  a  product  development 
process  affect  the  ability  to  attract  and  satisfy  customers.  These  are  lead  time,  total 
product  quality,  and  performance.  Lead-time,  or  design  cycle  time,  is  defined  as  the  time 
it  takes  a  company  to  move  from  concept,  to  production,  and  finally  to  market.  Lead- 
time  affects  the  design  and  market  acceptance  of  the  design.  Total  product  quality  is 
defined  as  the  extent  to  which  the  product  satisfies  customers'  requirements.  Both 
objective  and  subjective  evaluations  affect  quality.  Objective  evaluations  could  be  such 
things  as  fuel  efficiency  or  other  performance  parameters.  Subjective  evaluations  could 
include  aesthetics  and  styling.  Product  development  affects  total  product  quality  at  two 
levels:  the  level  of  design,  called  design  quality;  and  the  firm’s  ability  to  produce  the 
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design,  called  conformance  quality.  The  third  dimension  of  development  performance  is 
productivity.  This  is  the  level  of  resources  required  to  take  the  project  from  concept  to 
commercial  product.  This  includes  hours  worked,  materials  used,  and  any  services  used 
during  product  development.  According  to  Clark  and  Fujimoto  (1991),  to  analyze 
product  development  performance  one  must  develop  measures  of  these  three  performance 
dimensions. 

Lead-time  can  be  measured  as  the  time  elapsed  between  the  beginning  of  concept 
development  and  market  introduction.  Development  lead-time  is  a  measure  of  how 
quickly  a  firm  can  perform  the  many  different  activities  that  must  be  accomplished  to 
advance  from  concept  to  market  introduction.  Because  some  of  these  activities  are 
conducted  in  parallel  it  is  more  difficult  to  measure  development  lead-time. 

Total  product  quality  is  based  on  external  evaluation  of  many  attributes.  Clark 
and  Fujimoto  (1991)  used  multiple  indicators  to  measure  total  product  quality.  These 
include  customer  evaluation  of  product  characteristics,  customer  satisfaction,  and  product 
reliability  surveys.  Total  product  quality  includes  conformance  quality  (how  well 
products  delivered  to  customers  conform  to  specifications),  and  design  quality  (the 
degree  to  which  product  designs  match  customer  expectations).  Customer  satisfaction 
surveys  can  be  used  as  an  indicator  of  quality. 

Trends  have  shown  that  product  design  processes  are  becoming  more  focused  on 
team  activities  through  new  management  techniques  that  rely  on  multi-functional 
teaming.  Within  this  type  of  environment  it  is  necessary  for  individuals  to  share 
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information  and  collaborate  in  the  decision  making  process.  The  next  section  discusses 
the  relationship  between  collaboration  and  the  nature  of  product  design. 

Collaborative  Nature  of  Product  Design 

The  concept  of  group  work  is  often  referred  to  as  collaborative  work.  The  term 
collaboration  refers  to  a  goal-orientated  process  of  two  or  more  individuals. 

Collaborative  work  requires  communication  and  problem  solving  (Dhar  &  Olson,  1989). 
Communication  refers  to  the  exchange  of  information  for  purposes  such  as  notification 
and  clarification.  Problem  solving  refers  to  processing  of  information  for  monitoring, 
negotiating,  and  decision  making.  Projects  are  generally  monitored  and  decisions  made 
to  modify  or  achieve  goals  as  the  project  evolves.  Planning,  monitoring,  negotiating,  and 
decision  making  are  all  basic  components  of  collaborative  work. 

The  importance  of  collaboration  or  group  activity  in  the  design  process  is 
becoming  more  and  more  apparent  in  today’s  business  environment.  The  industry  norm 
is  becoming  reliant  on  the  utilization  of  cross-functional  teams.  This  trend  is  documented 
in  both  concurrent  engineering  and  Computer  Supported  Cooperative  Work  (CSCW) 
literature. 

Concurrent  Engineering 

There  are  several  analogous  terms  in  the  literature  referring  to  the  strategy  of 
applying  downstream  product  development  functions  at  an  early  concept  level.  These 
terms  include:  concurrent  engineering,  concurrent  design,  integrated  product  and  process 
design,  and  simultaneous  engineering  (Nevins,  1992).  Juran  and  Gryna  (1993)  describe 
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concurrent  engineering  as  a  process  of  designing  a  product  using  all  inputs  and 
evaluations  simultaneously  and  early  in  the  design  process  to  ensure  that  internal  and 
external  customers’  needs  are  met.  These  investigators  also  believe  that  the  purpose  of 
concurrent  engineering  is  to  reduce  design  cycle  time,  prevent  quality  and  reliability 
problems,  and  reduce  costs.  One  of  the  basic  ideas  of  concurrent  engineering  is  to  create 
and  assemble  a  team  that  is  focused  on  developing  or  redesigning  a  product.  These  teams 
are  composed  of  people  from  various  functional  elements:  product  management, 
development,  engineering,  and  manufacturing  (Sansone  &  Singer,  1992).  Juran  and 
Gryna  (1993)  contend  that  concurrent  engineering  is  not  a  set  of  techniques  but  a 
conceptual  methodology  that  enables  all  who  are  impacted  by  the  design  to  have  early 
access  to  design  information  and  to  have  the  ability  to  influence  the  final  design  to 
identify  and  prevent  future  problems. 

Clark  and  Fujimoto  (1991)  observed  a  trend  in  the  early  1980’s  from  purely 
functional  forms  of  organizations  to  more  integrated  structures.  These  researchers  found 
that  the  straightforward  functional  organizations  of  the  1960’s,  had  all  incorporated 
formal  mechanisms  for  cross-functional  coordination  by  the  late  1980’s.  In  this  later 
period,  the  principal  job  of  many  engineers  working  in  these  new  organizational 
structures  was  to  link  one  department  with  one  or  more  related  departments  --  called 
liaison  engineering.  Meetings  were  a  necessary  mechanism  for  information  exchange  and 
communication,  and  small  teams  were  established  around  particular  problems  or 
products.  Clark  and  Fujimoto  (1991)  also  discuss  a  type  of  integration  that  never  appears 
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in  formal  organizational  charts,  yet  is  vital  to  product  integrity  --  informal  face  to  face 
contact  between  working  engineers  and  managers.  As  a  result  of  the  evaluation,  the  use 
of  multifunctional  teams  is  commonplace  in  today's  business  environment. 

Communication  is  the  key  activity  for  group  work.  Floyd  and  Turner  (1989) 
describe  how  group  work  differs  from  individual  work  that  supports  the  need  for 
communication.  First,  group  work  involves  extensive,  person-to-person 
communications.  Second,  group  processes  need  to  be  supported  by  roles,  protocols,  and 
procedures.  Third,  a  task  management  function  is  required  to  monitor  individual 
contributions.  Lastly,  the  relationship  of  the  group  to  the  organization  must  be 
considered.  In  this  way,  organizational  culture,  norms,  power,  authority,  and  values  are 
integrated. 

Stickleym,  Evbuomwan,  and  Sivaloganathan  (1994)  expounded  that  human 
interaction  has  become  more  important  at  the  individual  and  management  levels.  They 
believe  that  organizations  are  moving  toward  a  cooperative  organizational  culture.  Not  all 
design  work  involves  teams,  but  a  team  setting  is  required  for  brainstorming,  discussion, 
consensus  decision  making,  presentations,  reporting  results,  giving  recognition,  and 
communication.  Team  members  are  also  a  needed  source  of  help  and  support  to  each 
other. 

Concurrent  engineering  requires  a  change  and  awareness  between  all  people  and 
functions  within  an  organization.  As  an  example  of  an  innovative  concurrent  engineering 
management  technique,  AT&T’s  Consumer  Products  Group  restructured  their 
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development  cycle  into  three  phases:  definition,  development,  and  production.  During  the 
definition  phase  functional  team  members  are  required  to  sign  the  initial  specification  as 
a  gesture  of  their  personal  commitment  (Sansone  &  Singer,  1992).  This  formal 
agreement  brought  functional  groups  together  in  the  early  stages  of  a  product 
development  effort.  Many  companies  have  found  that  they  do  not  need  to  make  large 
investments  to  begin  the  process;  they  only  need  to  invest  in  appropriate  training  that 
teaches  people  how  to  communicate  and  cut  across  organizational  barriers.  Concurrent 
engineering  methods  have  encouraged  the  exchange  of  information  between  functional 
groups  to  improve  the  quality  of  decisions  made  during  product  development. 

Some  research  has  been  conducted  on  expanding  the  concurrent  engineering 
philosophy  into  a  truly  shared  world  of  information  (Toye,  Cutosky,  Leifer,  Tenebaum.  & 
Glicksman,  1994;  Maxfield,  Fernando,  &  Dew,  1995).  These  ideas  are  based  on  the 
interconnection  and  share  distributed  information  through  standard  PC  workstations  --  an 
idea  called  collaborative  virtual  concurrent  engineering.  Effective  concurrent 
engineering  practices  require  that  functional  teams  have  access  to  knowledge  regarding 
product  versions,  customers,  and  design  data.  Special  knowledge  sources  have  to  be 
shared  and  coordinated  if  successful  product  design  is  to  be  accomplished  (Subrahmanian 
et  al.,  1993).  Automated  processes  can  assist  in  this  coordination,  sharing  effort. 

Toye  et  al.  (1994)  studied  how  to  apply  technologies  to  help  design  teams  gather, 
organize,  reassess,  and  communicate  design  information.  These  researchers  define  team 
design  as  a  shared  understanding  of  the  domain,  requirements,  product,  design,  and 
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design  process.  The  shared  understanding  is  developed  as  each  team  member  develops 
an  understanding  of  their  responsibilities  and  need  to  provide  information  to  other  team 
members.  The  process  involves  communication  and  negotiation.  The  researchers’ 
system  called  SHARE  enables  engineers  to  participate  as  a  team  using  their  tools  and 
databases.  SHARE  contains  six  features:  familiar  displays,  such  as  on-line  requirement 
documents;  collaborative  services,  such  as  desktop  video  conferencing;  on-line  ordering 
and  fabrication  services,  such  as  shipping  information;  specialized  services,  such  as 
simulation  tools;  a  distributed  data  management  system;  and  an  integration  infrastructure 
that  allows  heterogeneous  design  tools  in  the  environment. 

Maxfield,  Fernando,  and  Dew  (1995)  developed  a  system  called  Distributed 
Virtual  Engineering  (DVE)  which  supports  collaboration  among  members  of 
geographically  disperses  teams  engaged  in  concurrent  engineering.  DVE  supports  and 
environment  which  allows  a  team  to  interact  and  make  decisions  from  multiple 
perspectives  in  a  shared  information  space  using  accurate  virtual  prototypes  of 
mechanical  components.  This  system  uses  a  library  of  shared  objects  that  can  be 
populated  with  new  product  information  and  uses  a  video  conferencing  system  to  support 
communication  between  members  of  the  product  development  team. 

Researchers  in  the  field  of  CSCW  have  investigated  the  use  of  automation  on 
group  activities.  Automated  technologies  include  email,  videoteleconferencing,  and  more 
recently  the  use  of  virtual  environments  in  collaborative  group  efforts.  The  next  section 
discusses  CSCW  and  its  contribution  to  product  design. 
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Computer  Supported  Cooperative  Work  (CSCW) 

CSCW  emerged  in  1984  from  the  growing  interest  of  product  developers  in 
supporting  networked  groups  and  from  a  common  interest  among  various  researchers  as 
to  how  people  work.  There  are  several  analogous  terms  in  the  literature  associated  with 
the  term  CSCW.  These  are:  technology  for  teams,  computer-aided  teamwork,  groupware, 
and  workgroup  computing.  CSCW  is  an  interdisciplinary  approach  with  two  major  fields 
strongly  influencing  its  growth:  human  computer  interaction  and  computer  science. 
Applications  evolved  from  ideas  in  office  automation  including  desktop  conferencing, 
videoconferencing,  collaborative  authorship,  and  email  (Grudin,  1994;  Wilson,  1994). 

Wilson  (1994)  identifies  a  range  of  CSCW  activities,  including  supporting 
varying  sizes  of  groups  from  two  people  to  large  organizations,  supporting  both  face  to 
face  and  dispersed  cooperation,  and  supporting  real-time,  asynchronous  communications. 
The  researcher  further  divides  CSCW  into  two  components:  technology  and  human.  The 
technology  component  is  further  broken  into  four  categories:  communications  systems, 
shared  workspaces,  shared  information,  and  group  activity  support.  The  human 
component  is  also  divided  into  four  categories:  individual,  organizational,  group  work 
design,  and  group  dynamics.  CSCW  touches  on  a  wide  range  of  activities  that  involve 
groups  of  people  working  together  with  help  from  computer  technology. 

Scrivener  and  Clark  (1994)  identify  a  four-category  classification  strategy  for 
describing  CSCW  systems.  These  investigators  believe  that  CSCW  systems  vary  in  two 
principals:  their  mode  of  interaction,  and  the  geographic  distribution  of  users.  Mode  of 
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interaction  can  be  asynchronous,  occurring  at  different  times;  or  synchronous,  occurring 
at  the  same  time.  Geographic  distribution  can  be  either  local,  meaning  users  are  located 
in  the  same  environment  or  remote,  meaning  that  they  are  at  different  locations. 
Shneiderman  (1993)  supports  this  idea  and  refers  to  this  distinction  as  the  time-space 
matrix.  Examples  of  this  classification  identified  by  the  latter  researcher  are:  class  rooms 
and  meeting  places  are  considered  same  time  and  same  place;  shared  editors  are  same 
time  and  different  place;  project  scheduling  tools  are  different  time  and  same  place;  and 
lastly  email,  bulletin  boards  are  different  time  and  different  place. 

CSCW  researchers  have  identified  some  problems  with  multi-users  sharing  the 
same  workspace.  Some  of  these  problems,  as  described  by  Broil  (1995),  are  the 
distribution  of  objects  and  information  plus  the  delegation  of  rights  and  the  representation 
of  group  structure.  Also,  according  to  Wexelblat  (1993),  CSCW  programs  must  provide 
the  capability  to  exchange  objects,  to  vary  views  of  information,  and  to  coordinate  group 
interactions. 

Damton  (1995)  has  identified  five  categories  to  describe  ways  in  which  people 
work  together.  The  first  is  a  common  task  with  people  working  in  a  group  to  produce 
some  product.  As  an  example,  a  group  of  different  specialists  may  collaborate  to  design 
a  car.  Second,  is  sequence  of  tasks,  where  people  collaborate  in  a  chain  of  activities. 
Third,  is  problem  solving,  where  multi-disciplinary  teams  and  multi-skilled  teams  work 
together  to  solve  various  problems.  Fourth,  is  the  concept  of  command  and  control,  for 
example  in  air  traffic  control.  Fifth,  is  mutual  aid  or  reliance  of  people  for  mutual  benefit, 
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as  in  a  customer-supplier  relationship.  This  description  of  how  and  why  people  work 
together  can  be  directly  associated  with  scenarios  involved  in  the  product  design  and 
development  process  and  perhaps  in  a  VE  environment.  As  organizations  continue  to 
strive  towards  collaborative  interaction  with  customers,  suppliers,  and  other  relevant 
participants  in  an  organization,  the  field  of  CSCW  will  continue  to  evaluate  ways  in 
which  people  interact  and  cooperatively  work  (Damton,  1995). 

CSCW  is  based  on  conventional  user  interface  devices  (mouse,  pointer,  etc.),  but 
according  to  Takemura  (1992)  cooperative  work  using  virtual  environments  offers  more 
flexibility  and  offers  the  user  an  enhanced  sense  of  belonging  and  true  sharing  of 
resources.  Prior  to  the  emergence  of  VEs,  however,  most  multi-user  systems  gave  each 
user  the  illusion  that  he  or  she  was  the  only  one  using  the  computer  tool  thus  providing  an 
indirect  means  of  sharing  information.  Wexelblat  (1993)  believes  that  VEs  provide  a 
more  natural,  intuitive  approach  to  sharing  data  where  a  user  can  go  back  and  directly 
indicate  objects  through  gestures  and  conversion,  thus  making  interactions  more  smooth 
and  efficient  as  they  are  in  the  real  world. 

The  need  for  the  development  of  collaborative  design  tools  and  collaborative 
concurrent  engineering  tools  has  risen  from  the  changes  occurring  in  the  product  design 
process  towards  multi-functional  design  team  interactivity.  In  today’s  design 
environment,  people  are  bombarded  with  an  overabundance  of  information  regarding  the 
product,  the  customer,  the  system,  the  process,  and  its  suppliers.  A  need  exists  for  a 
holistic  approach  whose  foundation  is  in  automation  of  the  process. 
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Computer  Assistance  in  Product  Design  —  Emerging  Technologies 


This  section  discusses  current  computer  aided  design  technologies,  their  effect  on 
performance,  and  their  relevance  to  collaborative  team  design.  It  then  transitions  into  a 
new  emerging  technology  that  has  the  capability  of  enhancing  product  design  —  virtual 
environments. 

Computer  Aided  Design  (CAD) 

The  automation  of  the  design  process  through  the  development  of  computer  aided 
design  (CAD)  tools  began  in  the  early  1950’s  and  bloomed  in  the  1 980’s  (Barfield  et  al„ 
1987).  CAD  remains  a  major  tool  in  today’s  design  world. 

The  term  CAD  describes  any  system  that  uses  computers  to  assist  in  the  creation 
and  modification  of  a  design.  The  goal  for  creating  this  tool  was  to  augment  the 
creativity  and  decision  making  ability  of  the  designer.  A  typical  CAD  system  consists  of 
a  graphics  display  terminal,  a  digitized  tablet,  a  keyboard,  a  plotter  and  a  local  graphics 
processor.  The  system  can  be  classified  according  to  size  and  processing  capability  of 
the  hardware:  microcomputer,  workstation,  minicomputer,  or  mainframe  computer.  The 
tool  has  two  distinct  features:  it  allows  a  designer  to  create  an  object  and  to  perform  an 
engineering  analysis  directly  and  interactively  on  the  object. 

CAD  has  significantly  reduced  the  design  cycle  time  for  many  products  because 
designers  are  provided  with  a  set  of  tools  that  enhance  their  performance.  CAD  has  been 
documented  as  saving  up  to  70%  of  a  total  manual  design  time  (Barfield  et  al,  1987). 
CAD  technology  has  made  it  possible  for  product  designers  to  develop  an  assortment  of 
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options  when  designing  new  products.  Switching  to  a  computer-based  process  increased 
the  amount  of  possible  designs  that  could  be  considered  and  allowed  designers  to  develop 
many  design  iterations  in  a  reasonably  short  time.  Chang,  Joshi,  and  Hoberecht  (1992) 
identify  several  advantages  of  CAD.  They  are:  increased  productivity  of  the  design 
process,  improved  design  quality  and  accuracy,  reduced  development  and  testing  time, 
and  better  design  documentation. 

An  example  of  the  product  development  community’s  movement  towards  full 
usage  of  CAD  can  be  found  in  the  recent  Boeing  777  jetliner  project.  This  jetliner 
product  development,  described  by  Norris  (1995),  has  its  foundation  in  advances  in 
electronics,  CAD,  manufacture,  and  simulation.  It  is  the  first  commercial  jetliner  to  be 
100%  designed  on  a  computer  and  is  Boeing’s  first  truly  digital-designed  plane  —  a 
development  that  moved  directly  from  the  computer  screen  to  production. 

Pratt  (1994)  provides  a  historical  overview  of  CAD  tool  development.  The  tool 
was  originally  developed  to  automate  the  drafting  process  of  a  single  designer.  These  first 
systems  yielded  2-D  engineering  drawings  and  provided  a  means  for  the  generation  of 
traditional  drawings.  The  next  major  change  came  with  the  introduction  of  3-D  wire 
frame  models.  Wire  frame  models  provided  a  unified  model  of  the  object  rather  than  the 
several  partial  models,  as  is  the  case  of  the  traditional  three  orthogonal  views  in 
engineering  drawing.  One  advantage  of  wire  frame  models  is  that  the  computer  can 
automatically  generate  drawings  from  any  point  of  view  and  in  any  projection  chosen  by 
the  user.  The  next  development  was  the  solid  modeler,  which  brought  together  the 


51 


benefits  of  wire  frame  and  surface  models.  These  systems  contain  information  concerning 
all  faces  of  the  object,  including  the  surface  they  lie  on  and  the  edge  curves  that  bind 
them.  Also  stored  is  topological  information  indicating  how  all  these  elements  are 
connected. 

Today,  new  development  in  CAD  includes  parametric  modeling,  constraint-based 
modeling,  and  feature-based  modeling.  Parametric  modeling  allows  the  design  of  a 
product  in  which  certain  dimensions  are  not  fixed,  but  can  be  varied.  This  approach  is 
based  on  bi-directional  associativity  of  design  elements  (Engineering  Automation  Report, 
1995).  Constraint-based  modeling  builds  constraints  into  the  system.  Once  a  constraint  is 
defined  they  are  required  to  hold  even  when  any  design  modifications  are  made.  Feature- 
based  modeling,  include  features  that  are  a  local  geometric  configuration  on  the  surface 
of  a  manufactured  part  that  has  some  engineering  significance.  These  approaches  are  the 
first  step  towards  developing  a  design  tool  that  could  support  multiple  persons  working 
on  the  same  part  or  assembly  —  collaborative  CAD. 

Product  data  management  systems  play  a  key  role  in  achieving  today’s 
collaborative  CAD  capabilities.  Gear  (1996)  describes  these  systems  as  being  either  a 
check-in  or  check-out  types,  i.e.  data  management  system  monitors  the  configuration  of 
the  design  through  a  comparison  and  checking  procedure.  For  example,  two  people  can 
work  on  the  same  part  or  assembly.  Each  would  open  or  create  a  file  for  the  part  they  are 
designing.  Each  designer  could  edit,  modify  or  create  the  design.  This  work  would  be 
completed  independently,  then  stored  in  individual  files  with  different  names.  The 
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product  data  management  system  knows  that  two  individual  people  have  updated  the 
part.  The  next  step  requires  the  integration  of  the  work  of  the  independent  designers. 
When  the  part  is  called  up,  the  user  is  informed  that  two  part  files  exist.  The  data 
management  program  then  shows  all  the  changes  made  by  the  two  designers.  The  system 
then  progresses  to  ask  the  user  to  make  decisions  on  each  modification  made,  and  which 
modifications  should  be  kept  and  which  should  be  eliminated.  The  integration  process  is 
not  automatic,  but  requires  a  decision-maker  to  confirm  which  modifications  or  additions 
are  appropriate. 

The  previous  description  of  collaborative-CAD  identifies  some  of  the  limitations 
associated  with  current  CAD  capabilities.  As  the  design  changes  towards  more 
dependence  on  collaboration,  new  design  tools  will  emerge  and  new  strategies  for  group 
interaction  will  be  investigated— especially  if  companies  are  to  remain  competitive. 
Exploring  new  technologies  that  may  contribute  to  shorter  design  times  and  lower  design 
costs  are  continuously  necessary.  One  such  emerging  technology  is  virtual  environments. 

The  next  section  describes  the  value  of  virtual  environments  by  providing  the 
reader  with  a  brief  description  of  how  VE  systems  work,  differences  between 
monoscopic  and  stereoscopic  vision,  human  factor  research  on  3-D  displays,  and  product 
design  applications. 
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Virtual  Environments  (VEs) 


General  Description 

There  are  several  analogous  terms  in  the  literature  referring  to  virtual  environment 
(VE)  technology.  These  include:  virtual  reality  (VR),  telepresence,  enhanced  reality,  or 
synthetic  environments  (SE)  (Burdea  &  Coiffet,  1994).  These  investigators  also  define 
VR  as  a  high-end  user  interface  that  involves  real-time  simulation  and  interactions 
through  multiple  human  sensorial  channels  or  modalities  —  visual,  auditory,  and  tactile. 

In  different  but  related  definitions,  others  describe  VEs  as  synthetic  sensory  experiences 
that  communicate  physical  and  abstract  components  to  a  human  operator  (Kalawsky, 
1993),  or  an  immersive,  interactive  experience  based  on  real-time  3-D  graphic  images 
generated  by  the  computer  (Stuart,  1996).  VE  technology  has  the  capability  of  providing 
real-time  customer  feedback  and  interaction  necessary  for  successful  product  evaluation. 
This  technology  allows  a  person  to  experience  phenomena  that  appear  to  be  real  but  exist 
only  in  the  computer.  A  user  or  designer  can  be  immersed  into  the  concept  to  get  a 
realistic  visualization  of  the  final  product. 

VR  allows  the  designer  or  user  to  go  one  step  further  and  truly  experience  the 
design.  For  instance,  an  automotive  designer  could  sit  in  an  automobile  and  visualize  the 
interior  and  layout.  Then  the  designer  could  navigate  in  the  virtual  world  around  the  car 
and  look  at  exterior  qualities  of  the  vehicle  (Kalawsky,  1993).  This  technology  provides  a 
new  approach  to  exploring  reality  and  extends  our  senses  so  we  can  animate  products  to 
study  how  they  behave.  One  can  fly  inside  it  and  watch  the  vehicle  vibrate  as  it  moves 
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across  a  terrain  (Pimentel  &  Teixeira,  1995).  This  type  of  tool  can  be  useful  and  even 
invaluable  for  ergonomic  layouts  allowing  a  designer  to  make  extensive  changes  in  these 
virtual  concepts  to  insure  they  meet  their  customer’s  needs. 

The  incorporation  of  a  3-D  CAD  data  files  into  a  virtual  world  allows  for  visual 
evaluations.  There  are  problems  associated  with  file  translation  from  standard  CAD 
formats  to  formats  compatible  in  virtual  environments.  CAD  data  files  are  much  more 
complex  than  required  for  virtual  environment  application,  and  may  contain  several 
million  polygons  that  are  a  burden  in  a  VE  system  because  each  polygon  must  be 
processed  and  rendered  in  real-time.  Due  to  the  lack  of  maturity  of  the  technology,  VEs 
are  not  being  considered  as  the  next  generation  CAD  tool.  Instead  VEs  are  being  used  to 
aid  in  fleshing  out  initial  concepts,  analyzing  stress,  aerodynamics,  checking  ergonomics, 
designing  for  maintainability,  and  taking  designs  to  virtual  test  grounds  (Gottschalk  & 
Machlis,  1994).  These  systems  are  not  something  from  which  detailed  engineering 
information  can  be  derived,  but  VEs  provide  the  capability  to  rotate  and  look  at  objects  in 
a  3-D  perspective. 

Utilizing  VEs  in  the  design  phase  of  the  development  process  can  increase  design 
flexibility  by  allowing  the  exploration  of  various  alternatives.  It  provides  a  worthwhile, 
low-cost  approach  to  evaluate  products  prior  to  physical  prototype  builds.  Significant  cost 
savings  can  be  attributed  to  the  fact  that  problems  would  be  identified  and  corrected  in 
the  early  stages  of  development  before  any  commitment  is  made.  This  system  also  brings 
users  into  the  design  process  much  earlier,  and  creates  a  common  platform  and 
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environment  that  is  capable  of  multi-user  participation  by  an  entire  design  and  evaluation 
product  team. 

In  1995,  the  National  Research  Council  (NRC)  created  a  Committee  on  Virtual 
Reality  Research  and  Development  and  charged  them  with  a  goal  to  “recommend  a 
national  research  and  development  agenda  in  the  area  of  virtual  reality”  which  would 
guide  government  research  and  development  over  the  next  generation.  The  committee 
concluded  that  four  application  domains  show  the  most  promise.  They  are:  design, 
manufacturing,  and  marketing;  medicine  and  health  care;  hazardous  operations;  and 
training.  The  committee  recommended  that  the  design  function  be  one  of  the  principal 
focus  areas  of  VE  development  and  testing.  The  committee  also  recommended  that  the 
federal  government  provide  funding  for  a  program  aimed  at  developing  networking 
capabilities  and  standards  for  large  distributed  VEs.  Used  by  the  committee  was  the 
rationale  that  VEs  can  help  manufacturing  throughout  the  entire  product  development 
cycle.  For  example,  when  developing  design  requirements  VEs  can  serve  as  a  medium  in 
which  a  customer’s  mental  image  of  a  product  can  be  fashioned.  During  detailed  design 
VEs  allow  designers  to  reach  inside,  test  accessibility,  plan  maintenance.  Prior  to 
producing  a  product  VEs  can  create  pilot  manufacturing  lines  and  predict  product 
performance.  In  the  marketing  stage  VEs  can  provide  potential  customers  with  the  ability 
to  visualize  various  uses  of  a  product. 

The  following  description  of  a  VE  system  is  presented  to  the  reader  as  a  basis  for 
understanding  the  value,  capability,  and  complexity  of  the  technology. 
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The  VE  System 

VE  systems  generate  and  display  a  virtual  world,  and  process  inputs  and  outputs 
of  data.  The  system  consists  of  two  major  components:  hardware  and  software. 

Hardware 

The  hardware  system  consists  of  a  main  processor  and  input/output  devices.  The 
user  interacts  with  the  virtual  world  through  various  input  devices,  and  the  virtual  world 
in  turn  responds  to  the  user's  actions  by  using  appropriate  output  devices.  Dani,  Fathallah. 
and  Gadh  (1994)  diagrammed  a  typical  VE  system.  A  modified  version  is  depicted  in 
Figure  2. 

Input  Devices 

An  input  device  is  a  piece  of  hardware  that  is  used  to  put  data  into  a  computer  and 
allow  the  participant  to  interact  with  the  virtual  world.  Dani  et  al.  (1994)  categorized 
input  devices  into  five  equipment  types:  position  tracker,  digitizer  (3-D  mouse),  glove, 
biocontrollers,  and  voice  input.  These  devices  allow  a  person  to  pick  up  and  move  objects 
while  navigating  in  a  virtual  world,  or  they  allow  a  person  to  change  position,  viewpoint 
and  field  of  view  within  the  simulation  (Eddings,  1994).  They  also  sense  a  participant’s 
physical  position  and  orientation,  translating  them  into  corresponding  images  in  the 

I 

virtual  world. 
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Figure  2.  Typical  VE  system  (Modified  from  Dani  et  al.,  1994). 


Position  Tracker 

Position  trackers  are  used  to  track  the  position  and  orientation  of  the  user.  The 
state  of  the  technology  has  been  focused  on  tracking  the  position  and  orientation  of  the 
users  head  and/or  hand.  Currently,  tracking  systems  are  based  on  mechanical,  magnetic, 
acoustic,  optical,  or  inertial  technologies  (Dani  et  al.,  1994;  Stuart,  1996;  Haas,  1998). 

Mechanical  tracking  systems  physically  connect  the  user  to  the  tracker  with 
jointed  linkages  and  consequently  mechanically  measure  the  user’s  position.  Haas  (1998) 
describes  one  of  the  first  systems  developed  by  Sutherland  in  1968  as  a  shaft-like 
mechanical  assembly  that  hung  from  the  ceiling  and  attached  to  a  head  mounted  display. 
The  researcher  indicates  that  the  design  resembled  an  automobile  drive  shaft  and  was 
often  referred  to  as  the  “Sword  of  Damocles”  because  it  hung  precariously  over  the  users 
head.  Disadvantages  of  mechanical  systems  identified  were  that  they  were  heavy  and 
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uncomfortable  to  use.  While  mechanical  trackers  are  precise  and  responsive,  their 
mechanical  nature  restricts  motion  and  are  more  suited  for  telerobotic  applications  or  use 
in  cases  where  user  mobility  is  not  a  problem  (Haas,  1998). 

Magnetic  systems  rely  on  processing  the  behavior  of  magnetic  fields  to  calculate 
the  position  and  orientation  of  the  user.  Currently  two  commercial  types  of  magnetic 
trackers  are  available.  One  type,  offered  by  Polhemus  (1998),  uses  alternating  current 
(ac)  and  the  other,  Ascension  (1998),  uses  direct  current  (dc)  (Haas,  1998).  These  tracker 
systems  are  composed  of  a  magnetic  emitter,  a  magnetic  receiver,  and  a  controller  unit. 
For  a  detailed  description  of  how  these  systems  operate,  the  reader  is  referred  to  Haas 
(1998). 

Acoustic  tracking  systems  use  ultrasonic  frequencies  to  track  user  position. 
Frequencies  above  20kHz  are  used  so  that  emitter  sounds  are  not  heard  by  persons 
working  in  the  environment  (Haas,  1998).  Haas  describes  two  basic  approaches  to 
acoustic  tracker  design:  time-of-flight  (TOF)  measurement  and  phase-coherent  (PC) 
measurement.  The  TOF  measurement  approach  calculates  position  and  orientation  by 
using  multiple  emitters  and  sensors  and  measuring  the  elapsed  time  of  an  acoustic  wave. 
The  PC  measurement  approach  determines  distance  by  comparing  the  phase  of  a 
reference  signal  to  the  phase  of  an  emitted  signal  detected  by  sensors.  This  type  of 
tracking  system  can  generate  high  data  rates  because  phase  can  be  measured 
continuously,  which  leads  to  improved  accuracy,  responsiveness,  range,  and  robustness. 
This  type  is  more  conducive  for  larger  volume  environments. 
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Optical  tracking  systems  use  light  emitters  and  detectors  to  calculate  user 
position.  Currently  there  are  three  optical  tracking  systems  that  have  been  implemented 
in  VEs.  They  are  image  processing,  laser  ranging,  or  beacon  systems.  Image  processing 
types  of  systems  are  based  on  pattern  recognition  comparisons  between  known  and 
sensed  patterns.  Haas  (1998)  identifies  that  image  processing  systems  are  inherently  less 
accurate  as  the  distance  from  the  sensor  to  remote  object  increases.  Laser-ranging 
systems  measure  position  by  transmitting  laser  light  onto  a  remote  object  and  sensing  the 
reflected  light.  The  reflected  light  is  sensed  using  a  camera  that  sees  a  diffraction  pattern 
on  the  object.  Measure  of  distance  is  made  from  pattern  distortions  on  the  objects. 

Beacon  systems  rely  on  a  known  distance  between  either  emitters  or  sensors  to  determine 
position.  This  type  is  also  referred  to  as  fixed-transducer.  These  systems  are  the  most 
common  form  of  optical  trackers.  Regardless  of  type,  optical  trackers  can  suffer  from 
ranging  errors  caused  by  spurious  light,  ambiguity  of  surface,  and  occlusion.  In  contrast, 
their  positive  attributes  include  high  data  rates  and  good  responsiveness  (Haas,  1998). 

Inertial  tracking  systems  use  gyroscopes  to  sense  the  orientation  of  an  object  in 
space.  These  systems  measure  changes  in  pitch,  yaw,  and  roll.  Haas  (1998)  comments 
that  these  systems  are  more  robust  because  they  make  direct  measurements  of  angular 
change.  Inertial  trackers  are  immune  to  the  interference  experienced  when  using 
magnetic  or  acoustic  tracker  systems.  The  disadvantage  of  this  type  of  tracker  is  that 
small  inexpensive  commercial  solid-state  gyros  are  less  accurate  (Haas,  1998). 
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Digitizer 

A  digitizer  includes  devices  such  as  a  six-degree  freedom  mouse,  trackballs,  and 
joysticks.  Dani  et  al.  (1994)  observes  that  these  devices  have  evolved  from  traditional 
mouse/trackball  technology  to  a  more  advanced  form  of  data  input.  They  are  used  to 
allow  a  user  to  interactively  select  coordinate  positions  of  an  object  (Hearn  &  Baker, 
1997)  and  to  navigate  through  a  virtual  world. 

Glove 

Instrumented  gloves  are  often  refereed  to  as  datagloves,  virtual  hand  controllers, 
or  hand-gesture  interface  devices  (Stuart,  1996).  These  devices  can  be  used  to  grasp  a 
virtual  object  or  make  gestures.  They  can  be  worn  over  the  hand  like  a  normal  glove,  but 
contain  sensors  that  are  used  to  measure  the  position  of  the  hand  (Hearn  &  Baker.  1997). 
These  devices  combine  bend  sensing  and  position  tracking  technologies  to  detect  posture 
and  position  of  the  hand;  and  function  to  capture  the  movement  of  the  fingers,  thumb  and 
wrist,  to  provide  input  signals  to  VE  systems.  Stuart  (1996)  discusses  some  approaches 
to  sense  bending  and  joint  flexion  capture.  The  DataGlove™  developed  by  VPL  uses 
fiber-optic  cables  attached  to  lycra  gloves.  The  amount  of  light  passing  through  the 
cables  is  a  measurement  of  joint  flexion.  When  a  joint  is  flexed  less  light  is  passing 
through  the  optic  cable.  Another  less  expensive  type  developed  by  Mattel  is  based  on 
measuring  the  strain  of  flexion  with  flexible  polyester  strips  covered  by  special  ink. 
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Biocontrollers 


Biocontrollers  are  devices  that  detect  and  process  bioelectrical  signals.  These 
signals  can  be  the  result  of  muscle  or  brain  activity  (Stuart,  1996).  Stuart  (1996)  defines 
the  term  myoelectric  as  muscle  movement  and  their  transmission  of  electrical  signals. 

The  researcher  suggests  that  myoelectric  biocontrollers  are  used  for  gaze  control 
interfaces  and  that  they  are  important  for  users  with  physical  handicaps.  Related  is  the 
term  cerebroelectric  which  refers  to  electrical  activity  associated  with  brain  signals. 

Voice 

Voice  input  systems  interact  with  the  VE  through  the  use  of  voice  or  speech 
recognition  technology.  These  systems  allow  the  user  to  interact  and  navigate  in  the  VE 
through  voice  command.  Speech  recognition  systems  can  be  categorized  as  speaker 
dependent  or  speaker  independent.  Speaker  dependent  systems  require  users  to  train  the 
system  prior  to  use.  These  systems  have  a  good  recognition  rate  because  they  only  deal 
with  a  single  voice.  In  contrast,  speaker  independent  systems  have  lower  recognition 
rates  because  they  have  to  detect  different  individual  voices.  However,  the  advantage  in 
this  type  of  voice  recognition  system  is  that  the  system  is  trained  only  one  time. 

Main  Processor 

The  main  processor  contains  the  processing  power  to  run  the  simulated  virtual 
world  and  produces  a  sense  of  reality.  The  main  processor  is  sometimes  referred  to  as  the 
reality  engine  because  it  is  an  essential  element  in  the  VE  sensory  feedback  loop.  The 
reality  engine’s  function  is  to  create  and  manage  the  interactions  in  a  virtual  environment. 
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and  perform  real-time  computations  that  produce  correct  system  performance.  Stuart 
(1996)  defines  real-time  image  generation  as  the  generation  of  images  at  quick  enough 
speeds  to  be  displayed  at  a  chosen  frame-rate.  In  VE  applications,  fast  processing  speed  is 
essential  because  it  is  necessary  to  have  the  physical  movements  of  a  user  synchronized 
with  movements  in  the  virtual  world.  It  is  also  important  for  users  to  accurately  perceive 
the  correct  and  appropriate  movement  responses  in  the  virtual  world  otherwise  they  may 
feel  nausea  or  suffer  other  negative  effects.  Main  processors  for  VE  application  range 
from  low-end  PC  based  systems  to  high-end,  high  performance  computing  stations. 

Output  Devices 

Human  brains  allow  individuals  to  perceive  the  exterior,  physical  world  through 
five  primary  senses:  sight,  sound,  touch,  smell,  and  taste  (Sekuler  &  Blake,  1994).  A  VE 
system  presents  a  simulated  world  to  the  human  senses  through  VE  output  devices.  These 
devices  are  the  primary  feedback  users  in  VEs  so  they  can  perceive  appropriate 
information  for  interacting  in  the  VE  and  perform  tasks.  Currently,  the  state  of  VE 
technology  output  devices  is  limited  to  the  sense  of  sight,  sound,  and  to  some  degree 
touch. 

Visual  Display  Devices 

Human  vision  is  considered  the  most  powerful  sensorial  channel  providing  a 
sense  of  sight.  Humans  are  capable  of  processing  extremely  large  bandwidths  of 
information  at  any  single  moment  in  time  without  conscious  effort.  Because  human 
vision  is  critical  in  providing  humans  necessary  information,  there  has  been  a  multitude 
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of  visual  display  developments  both  in  industry  and  academia.  The  most  common  visual 
display  systems  are  helmet-mounted  displays  (HMDs),  binocular  omni  orientation 
monitors  (BOOMs),  and  stereoscopic  glasses.  Other  visual  display  systems  such  as  the 
CAVE  Automatic  Virtual  Environment  (CAVE)  use  large-screen  projectors,  while  others 
use  holographic  optics  to  create  virtual,  visual  experiences. 

The  breadth  of  this  study  is  restricted  to  the  application  of  commercial  3-D  visual 
display  systems  on  the  CDR  process,  therefore  the  following  information  provides  the 
reader  with  a  general  understanding  of  commercially  available  3-D  visual  display 
systems.  Before  describing  these  systems  a  description  of  stereoscopic  and  monoscopic 
vision  is  provided  to  show  the  relationship  between  visual  display  systems  and  human 
depth  perception  —  a  human’s  ability  to  see  3-D  scenes.  More  specific  technical 
requirements  for  visual  display  systems  for  VEs  include  spatial  resolution,  depth 
resolution,  responsiveness,  field  of  view  (FOV),  storage  and  refresh  rate,  and  color 
(Stuart,  1996).  Each  3-D  visual  display  system  offers  different  technical  capabilities  and 
thresholds.  For  a  more  in-depth  understanding  of  these  technical  aspects,  the  reader  is 
referred  to  McKenna  and  Zeltzer  (1992)  who  provide  a  detailed  description  of  underlying 
technology  requirements  and  limitations  for  3-D  display  systems  and  performance 
comparisons  with  the  human  visual  system. 

Monoscopic  versus  Stereoscopic  Vision 

Visual  display  systems  are  often  categorized  by  how  the  observer  uses  his  or  her 
eyes  to  create  the  visual  image.  Monoscopic  systems  display  the  same  2-D  image  to  both 
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eyes.  When  using  these  systems,  depth  is  only  perceived  based  on  cues  inherent  in  the 
image  created.  These  cues  are:  shadows,  occlusions,  texture,  and  object  detail  (Burdea  & 
Coiffet,  1994).  Other  cues  identified  by  Wickens  (1990)  are  highlighting,  proximity- 
luminance  covariance,  linear  perspective,  occlusion,  and  height  in  the  visual  field.  In 
contrast,  stereoscopic  systems  present  different  images  to  the  each  eye,  creating  the  effect 
of  depth  (Wickens,  1990;  Sekuler  &  Blake,  1994;  Kalawsky,  1993;  Burdea  &  Coiffet. 
1994;  Diner  &  Fender,  1993).  Sekuler  and  Blake  (1994)  provide  a  description  of 
stereoscopic  vision.  In  their  description  it  is  pointed  out  that  the  anatomy  of  the  human 
eyes  plays  a  major  role  in  providing  humans  with  the  ability  to  see  depth.  Because  the 
eyes  are  located  in  the  front  of  the  head  and  are  laterally  separated,  they  view  objects 
from  slightly  different.  These  differences  or  disparities  between  the  view  seen  by  each 
eye  give  humans  the  ability  to  discriminate  extremely  small  differences  in  relative  depth. 
Ma,  Hollerback,  and  Hunter  (1993)  list  the  advantages  of  stereoscopic  vision  as 
producing  superior  ability  to  resolve  spatial  detail,  visual  detection,  and  form  recognition. 
HMD,  BOOM,  and  stereoscopic  glass  visual  display  systems  discussed  next  are 
considered  stereoscopic  visual  display  devices. 

Helmet  Mounted  Display  (HMD) 

Larijani  (1994)  defines  a  Helmet  Mounted  Display  (HMD)  as  a  headpiece  or 
head-held  brace  with  viewing  or  optical  devices  located  or  suspended  in  front  of  the 
user’s  eyes.  The  design  itself  places  screens  very  close  to  the  eye,  because  of  the  limited 
distance  special  optics  are  required  to  allow  the  eyes  to  focus  at  such  a  short  distance. 


65 


Special  optics  are  also  used  to  magnify  the  screen  image  in  order  to  fill  the  eye’s  FOV 
(Burdea  &  Coiffet,  1994).  The  physical  design  of  an  HMD  takes  into  account  the  head 
geometry  of  the  user  (Stuart,  1996).  Today  most  HMDs  are  based  on  either  cathode  ray 
tube  (CRT)  technology  or  liquid  crystal  display  (LCD)  technology  (Burdea  &  Coiffet. 
1994).  HMDs  based  on  small  LCDs  are  relatively  cheap  but  have  low -resolution 
capability.  In  terms  of  volume  of  sales,  most  HMDs  are  CRT  based  (Kalawsky,  1993). 
CRTs  have  the  advantage  of  higher  resolution  than  their  LCD  competitors,  but  CRT 
based  HMDs  carry  the  added  burden  of  weight.  Adam  (1994)  comments  that  the  CRT 
display  device  increases  the  weight  and  volume  of  the  HMD.  There  are  ongoing  research 
activities  aiming  at  the  elimination  of  CRTs  by  placing  the  display  directly  on  a 
microchip.  This  technology  researched  by  Texas  Instruments  is  referred  to  as  a  Digital 
Microminor  Device  (DMD)  (Burdea  &  Coiffet,  1994).  It  has  the  capability  of  improving 
resolution  and  decreasing  problems  associated  with  high  weight. 

The  importance  of  HMD  devices  are  that  they  have  the  capability  of  generating  3- 
D  images  and  providing  the  user  with  the  effect  of  being  fully  immersed  in  a  virtual 
world,  but  the  technology  is  limiting.  Singh,  Feiner,  and  Thalmann  (1996)  comment  that 
low-resolution  display,  small  field  of  views,  and  poor  ergonomics  hinder  current 
generation  HMDs.  Because  of  these  limitations,  these  researchers  believe  that  current 
HMD  technology  is  not  suited  for  precision  work  or  efforts  that  require  extended  use.  In 
addition,  these  researchers  point  out  that  insufficient  computing  power,  lack  of  good 
interaction  techniques,  and  high  development  costs  are  a  probable  cause  for  the  lack  of 
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adaptation  as  useful  tools.  Deering  (1996)  supports  these  statements  claiming  that  low 
resolution  and  extremely  distorted  optics  of  most  HMD  displays  make  them  unsuitable 
for  fine  manipulation  of  3-D  objects.  The  researcher  further  believes  that  HMD 
technology  is  limited  to  simple  positioning  tasks  where  objects  are  already  created 
outside  the  VE.  Holmgren  and  Robinett  (1993)  claim  that  VR  applications  require  a 
display  device  that  are  lightweight,  small,  and  provide  high  resolution.  The  weight 
requirement  is  needed  for  reasons  of  comfort  for  long  use.  These  researchers  suggest  that 
an  ideal  HMD  should  resemble  a  pair  of  sunglasses.  Pimental  and  Teixeira  (1995) 
provide  a  resource  guide  of  current  HMD  developers  and  suppliers. 

Binocular  Omni-Orientation  Monitor  (BOOM) 

A  BOOM  developed  by  Fakespace  Laboratories  was  first  introduced  in  1988 
(Fakespace,  1998;  Kalawsky,  1993;  Stuart,  1996).  It  is  a  display  device  that  is  based  on 
mounting  a  viewer  on  a  stand  and  permitting  the  user  to  navigate  or  move  through  the 
virtual  world.  Users  bring  the  viewer  to  their  face  and  holding  it  there,  use  it  to  view  the 
virtual  environment.  The  device  is  a  mechanical  arm  that  supports  either  a  CRT  or  LCD 
based  imaging  system  at  one  end.  Boom  displays  have  a  six-degree  of  freedom  motion  (x, 
y,  z,  roll,  pitch,  and  yaw)  support  structure  (Fakespace,  1998).  The  system  uses  optical 
encoders  at  each  of  six  joints  on  the  support  structure  to  provide  high-speed,  high 
precision  information  regarding  position  and  orientation  of  the  display  as  the  viewpoint 
changes.  The  arm  of  the  system  is  counterbalanced  so  that  the  display  has  zero  weight. 

By  using  this  design,  problems  with  optics  and  display  weight,  and  display  size  are 


67 


eliminated  (Pimentel  &  Teixeira,  1995).  BOOM  systems  can  use  higher  resolution  CRT- 
based  displays  because  they  can  offset  their  additional  weight.  However,  the  improved 
resolution  and  performance  has  higher  associated  costs. 

The  design  of  a  BOOM  system  provides  users  with  ease  of  access.  Users  can 
easily  switch  between  the  VE  and  their  work  environment  without  having  to  get  in  and 
out  of  a  HMD  (Stuart,  1996).  The  BOOM  device  has  been  applied  to  a  variety  of 
research  projects  (Fakespace,  1998).  NASA  Ames  Research  Center  has  used  the 
technology  for  scientific  visualization  of  windtunnel  characteristics  (Bryson,  1993).  This 
device  has  been  used  for  CAD  and  concurrent  engineering  work  at  Ford  Motor  Company. 
Boeing  Computer  Services,  and  the  U.S  Army  TACOM.  Academia  has  explored  the 
technology;  one  example  is  the  University  of  Central  Florida’s  Institute  of  Simulation  and 
Training,  where  the  BOOM  is  used  for  training  military  hostage  rescue  teams. 

Stereoscopic  Glasses 

Stereoscopic  glasses  used  in  conjunction  with  a  display  monitor  are  an  alternative 
approach  to  3-D  visualization  that  permits  very  high  resolution  at  the  expense  of 
immersion.  Stereoscopic  glasses  are  also  referred  to  as  shutter  glasses.  They  are 
synchronized  to  the  alternating  display  of  two  separate  images  on  a  monitor.  When  the 
right  side  image  is  shown,  the  shutter  over  the  left  eye  is  closed,  and  the  one  over  the 
right  eye  is  opened  (Stuart,  1996).  When  the  second  image  is  shown,  the  left  eyes  shutter 
opens,  and  the  one  over  the  right  eye  closes.  Dani  et  al.  (1994)  describes  this  system 
where  two  images  of  a  virtual  scene  are  shown  alternately  at  a  very  high  rate  on  a  CRT 
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monitor.  The  rate  of  at  least  60  times  per  second  per  eye  is  required  for  the  brain  to 
interpret  the  resulting  image  as  a  single  stereoscopic  view  (Eddings,  1994).  When 
combined  with  tracking  technologies  one  can  determine  the  position  of  the  user  and  use  it 
to  display  viewpoint  dependent  images  on  the  display  monitor 

There  are  several  ways  to  shutter  the  glasses  including  manually  closing  and 
opening  the  shutters,  but  the  most  widely  used  technology  are  liquid  crystal  shutters.  By 
using  an  infrared  transmitter  one  can  coordinate  the  display  rate  to  the  frequency  with 
which  each  of  the  glasses  is  blacked  out  providing  the  user  with  the  perceived  3-D  image. 
Stereoscopic  glasses  are  a  lightweight,  wireless,  and  hands  free  viewing  device 
(Stereographies,  1998).  Stereoscopic  glasses  provide  low  cost  alternatives  therefore  are 
more  prevalent  in  the  entertainment  industry. 

The  stereoscopic  glasses  device  has  been  applied  to  a  variety  of  research  projects 
(Stereographies,  1998).  Researchers  at  Parke-Davis  Pharmaceuticals  are  using 
stereoscopic  glasses  system  to  visualize  complex  molecules  in  order  to  design  new  drugs. 
Designers  at  Ford  Motor  Company,  General  Motors,  and  Chrysler  Corporation  are  using 
the  technology  to  assist  in  the  design  and  evaluation  of  new  car  concepts.  Military 
researchers  at  the  U.S  Army  TACOM  are  using  the  technology  to  bring  the  user  in  the 
design  process  by  allowing  them  to  evaluate  new  concept  designs. 

Other  Visual  Display  Systems 

The  Cave  Automatic  Virtual  Environment  (CAVE)  system  was  originally 
developed  at  the  University  of  Dlinois-Chicago’s  Electronic  Visualization  Laboratory  in 
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1992.  Today  this  product  is  commercially  available  through  Pyramid  Systems, 
Incorporated.  The  CAVE  is  a  projection-based  VE  system  that  surrounds  the  user  with 
four  screens.  The  screens  are  arranged  in  a  cube  10  x  10  x  10  feet  made  up  of  three  rear 
projection  screens  for  walls  and  a  down  projection  screen  suspended  from  the  ceiling  for 
the  floor  (Eddings,  1994).  The  down  projector  overhead  points  to  a  mirror,  which 
reflects  the  images  onto  the  floor.  Graphics  are  projected  onto  the  screens.  A  viewer 
wears  stereoscopic  glasses  and  a  headtracking  device.  As  the  user  movers  inside  the 
CAVE,  the  correct  stereoscopic  perspective  projections  are  calculated  for  each  wall.  The 
image  moves  with  and  surrounds  the  user.  An  SGI  Onyx  computer  with  three  Reality 
Engine  graphic  pipes  controls  the  projected  images.  Each  reality  engine  is  attached  to  a 
CAVE  wall.  The  CAVE  was  originally  designed  to  be  a  useful  tool  for  scientific 
visualization  and  for  multiple  person  viewing.  Six  to  ten  people  can  view  the  projection, 
while  one  person  with  a  headset  and  joystick  computer  controls  the  simulation’s 
perspective.  In  the  CAVE  a  user  sees  his/her  own  body. 

Like  the  CAVE,  other  systems  based  on  projector  and  stereoscopic  glasses  have 
emerged  in  the  commercial  market.  These  include  the  ImmersaDesk™  developed  by 
Pyramid  Systems,  Inc.,  Immersive  Workbench™  by  Fakespace,  Inc,  Visionarium  by  SGI, 
and  various  powerwall  concepts.  The  first  two  systems  offer  a  stereoscopic  imaging 
system  designed  to  resemble  traditional  designer  drafting  boards.  The  Visionarium  uses  a 
180  degree  curved  screen  providing  users  with  the  sense  of  surrounding  themselves  in  the 
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VE.  The  powerwall  system  uses  a  20  x  8  feet  rear  projected  screen  to  create  life-like 
sized  virtual  images. 

Typical  projector-based  head  tracked  stereo  systems  have  only  been  able  to 
produce  a  single  head-tracked  image.  Recently,  Fakespace,  Inc  has  introduced  an 
additional  feature  to  their  Immersive  Workbench  called  the  Duo  system.  This  system 
allows  two  simultaneously  and  individually  head-tracked  stereo  pairs  to  be  displayed. 

This  system  allows  for  collaborative  activities,  where  two  people  can  be  immersed  in  a 
shared  workspace  (McDowell  &  Bolas,  1997;  Agrawala  et  al.,  1997).  It  is  postulated  that 
this  type  of  VE  system  is  appropriate  when  the  sense  of  an  objects  presence  is  needed  and 
when  one  wants  a  virtual  model  that  appears  to  sit  on  the  surface  of  a  workbench  (Bolas, 
Bryson,  &  McDowell,  1998).  Some  case  studies  using  the  Immersive  Workbench 
technology  have  been  explored.  These  include  jet  simulations,  molecular  docking, 
molecular  dynamic  simulation,  and  visualizing  unsteady  flow  above  a  destroyer  vessel. 

Other  research  has  explored  the  use  of  holographic  imaging  to  develop  3-D  virtual 
objects.  U.S.  Army  researchers  at  the  U.S.  Army  TACOM  have  developed  a  prototype 
Holographic  Imaging  system  that  combines  holographic  laser  technology  with  real-time 
computer  simulation  to  develop  the  capability  to  view  objects  in  3-D  (Bochenek  &  Buck, 
1997).  The  current  prototype  system  uses  a  high  performance  graphics  computer, 
projectors,  a  holographic  optical  element,  and  custom  software  to  create  the  3-D  imagery. 
These  3-D  objects  appear  to  float  just  in  front  of  the  holographic  optical  element  in  3-D 
space  (Keebler,  1995).  The  system  can  be  configured  in  either  a  high  or  low-resolution 


71 


version.  The  low-resolution  prototype  relies  on  using  LCD  projectors  and  the  high- 
resolution  prototype  uses  high  definition  projectors.  The  high  definition  projectors 
provide  greater  resolution  at  the  expense  of  losing  viewing  mobility.  The  prototype 
system  requires  the  viewer  to  be  in  a  stationary  location.  Research  plans  are  to  increase 
the  performance  of  the  prototype  system  while  maintaining  high  resolution. 

Audio  Devices 

The  sense  of  sound  is  the  second  most  important  sensorial  channel  for  virtual 
experiences.  The  sensation  of  sound  compliments  the  visual  modality  because  a  human 
can  process  both  visual  and  audio  information  in  parallel  (Sekuler  &  Blake,  1994). 
Gilkey  and  Weisenberger  (1995)  have  argued  that  sound  is  just  as  important  in  virtual 
environment  systems  because  sound  is  critical  in  achieving  maximum  sense  of  presence. 
Auditory  information  is  used  to  provide  the  user  feedback  about  his/her  interactions  in 
the  virtual  world.  When  sound  is  added,  the  overall  user  interactivity  and  immersion  are 
increased  (Burdea  &  Coiffet,  1994).  Three-dimensional  sound,  which  appears  to  come 
from  different  directions,  can  provide  a  more  realistic  virtual  experience.  In  VEs  the 
sound  position  changes  dynamically  in  real-time.  Few  vendors  develop  devices  that  can 
produce  realistic  3-D  sound  in  real  time.  Crystal  River  Engineering  has  been  in  the 
forefront  of  developing  commercial  3-D  audio  systems.  One  product,  the  Acoustron  II 
provides  a  full  spectrum  of  3-D  sound  to  include  Doppler  shifts,  spatialization,  and 
acoustic  raytracing  of  environments.  Inputs  to  the  system  include  sound  source  position 
and  orientation,  listener  position  and  orientation,  acoustic  environment  characteristics. 
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and  wall  reflection  and  transmission  attributes.  For  information  regarding  other  vendor 
sources,  model  availability,  technology  basis,  and  related  cost  the  reader  is  referred  to 
Pimental  and  Teixeira  (1995)  or  Stuart  (1996).  Each  provides  product  resource  guides  for 
current  sound  developers  and  suppliers. 

Tactile  Devices 

Tactile  or  haptic  feedback  provides  a  user  with  a  sense  of  touching  objects  in  the 
virtual  world.  There  are  a  variety  of  technologies  that  have  been  used  to  create  the  sense 
of  touch.  Stuart  (1996)  provides  a  comparative  evaluation  of  tactile  display  technologies. 
In  summary,  there  exist  five  technologies.  Vibro-tactile  displays  give  the  sense  of  touch 
via  vibration  on  the  fingertip.  Pneumatic  displays  create  the  impression  of  touch  by 
creating  pressure  with  air  ring,  jets,  or  pockets  that  touch  the  user.  Electrocutaneous 
displays  create  the  sense  of  touch  by  sending  tiny  pulses  of  electricity  through  electrodes 
touching  the  users  skin.  Shape  memory  alloys  change  shape  and  create  the  sense  of  touch 
through  their  pressure  on  the  skin.  Functional  neuromuscular  stimulation  creates  the 
impression  of  touch  through  direct  stimulation  to  the  neuromuscular  system 

Current  tactile  devices  according  to  Burdea  (1996)  are  limiting  and  do  not  give 
users  the  sense  of  weight,  surface  smoothness,  compliance,  or  temperature.  Burdea 
(1996)  feels  that  the  usefulness  and  realism  of  current  generation  VR  systems  are 
hampered  by  the  lack  of  force  and  tactile  feedback  to  the  user.  Kontarinis  and  Howe 
(1995)  offer  some  insight  into  the  usefulness  of  vibration  tactile  devices.  These 
researchers  suggest  that  tactile  information  may  be  useful  for  inspection  tasks  where  one 
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is  detecting  the  looseness  in  an  assembly  of  parts  or  smoothness  of  a  surface.  These 
devices  can  also  be  used  to  indicate  that  contact  between  an  object  has  occurred,  and  for 
simple  positioning  and  enhanced  sense  of  presence.  Burdea  (1996)  details  tactile 
feedback  interfaces,  haptic  sensing  and  controls,  human  factor  issues,  and  haptic 
feedback  applications.  Much  research  needs  to  be  conducted  to  develop  new  concepts 
and  products,  which  will  give  the  human  the  sensation  of  touch  in  virtual  space  thereby,  a 
more  realistic  experience  in  the  VE. 

Software 

VE  software  packages  are  used  to  build  virtual  worlds.  This  software  is  used  to 
design  the  landscape  and  objects  participant  encounters  in  a  virtual  world.  Bricken  and 
Coco  (1994)  classify  VE  software  development  systems  into  two  categories:  VR 
Toolkits,  or  integrated  software  systems. 

VE  toolkits  contain  libraries  of  program  functions  and  objects  that  let  a  world 
builder  create  an  application.  These  libraries  include  functions  for  importing  geometric 
objects  from  computer  aided  design  programs  that  generate  a  virtual  world.  They  also 
allow  the  designer  to  assign  behaviors  to  objects  in  the  virtual  world.  In  these  systems  C 
or  C++  programming  language  is  often  used.  This  type  of  tool  offers  tremendous 
flexibility  but  requires  programming  experience  to  use  it.  The  toolkits  have  built  in 
device  drivers  for  interfacing  with  commercial  input  and  output  devices  (Wang,  Green,  & 
Shaw,  1995)  and  built  in  rendering  functions  such  as  coloring,  texturing,  and  shadowing. 
Some  of  the  available  software  products  are  MR  toolkit,  created  by  the  University  of 
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Alberta;  VLib,  created  by  University  of  North  Carolina;  World  Toolkit,  created  by  Sense 
8;  Performer,  created  by  SGI;  and  Cyberspace  Developer’s  kit  for  linkage  with 
AutoCAD. 

Integrated  systems  are  intended  for  use  by  nonprogrammers.  These  systems  hide 
the  low-level  computational  details  by  providing  an  intuitive  visual  and  default  options 
for  constructing  events.  These  systems  also  provide  standard  software  facilities  required 
for  virtual  world  development  and  support  for  input  and  output  devices.  This  type  of 
system  allows  users  to  create  a  virtual  world  database  giving  objects  physical  and 
behavioral  properties.  These  include  object  geometry,  color,  texture,  motion,  collision, 
auditory,  and  haptic.  Some  of  these  systems  are  RB2  developed  by  VPL  Research  and 
dVS  from  Division  Ltd.  (Bricken  &  Coco,  1994). 

Immersion  and  the  Sense  of  Presence 

The  term’s  immersion  and  sense  of  presence  are  often  intermingled  in  discussions 
about  VEs.  For  this  reason  a  description  of  this  phenomena  is  provided  to  the  reader. 

VEs  convey  a  level  of  presence  to  the  user  within  the  synthetic  environment.  A  greater 
sense  of  presence  is  considered  a  desirable  outcome  for  VE  participants.  Kalawsky 
(1993)  describes  that  many  people  who  have  tried  VE  systems  have  claimed  that  they  felt 
as  though  they  were  “immersed’  or  “present”  in  the  VE.  Slater  and  Usoh  (1993)  define 
the  sense  of  presence  as  the  belief  that  people  using  VE  technologies  believe  they  are  in  a 
world  other  than  where  there  real  bodies  are  located.  Gilkey  and  Weisenberger  (1995)  list 
some  design  goals  that  have  been  suggested  to  provide  a  maximum  sense  of  presence. 
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These  goals  are: 

1 .  Users  should  receive  high  bandwidth  and  high  fidelity  sensory  information 
that  is  consistent  across  modalities. 

2.  The  visual  display  should  provide  users  a  wide  field  of  view. 

3.  Users  should  be  able  to  see  their  own  virtual  bodies  through  the  visual 
displays. 

4.  Users  should  be  able  to  move  sensors  and  control  viewpoint  with  the 
environment. 

5.  Users  should  be  able  to  change  the  environment. 

6.  The  virtual  environment  generation  system  should  be  responsive,  with 
minimal  internal  delays. 

7.  The  behavior  of  the  environment  would  be  orderly,  in  that  it  appears  to  obey 
some  set  of  user  predicable  laws. 

8.  Other  beings  or  objects  in  the  environment  should  respond  to  the  presence  of 
the  observer. 

9.  Users  should  be  isolated  from  the  real  world,  (pp.  357-358) 

Currently,  there  is  no  standard  metrics  that  defines  the  level  or  degree  of 

immersion  or  presence  conveyed  in  a  VE.  Sheriden  (1992)  notes  “that  presence  is  a 
subjective  sensation— a  mental  manifestation  that  is  not  so  amenable  to  objective 
physiological  definition  and  measurement."  Travis,  Watson,  and  Atyeo  (1994)  define 
this  lack  of  standards  regarding  immersion  as  the  great  immersion  debate.  They  indicate 
that  the  debate  is  about  what  the  systems  requirements  are  for  true  immersion.  These 
researchers  believe  that  vendors  of  HMDs  use  immersion  as  a  defining  feature  of  VR 
while  vendors  of  desktop  systems  are  more  liberal  in  their  definition.  Several  other 
researchers  indicate  that  great  emphasis  has  been  placed  on  creating  methods  for 
measuring  the  level  of  immersion  or  presence  within  a  VE.  Slater  and  Usoh  (1993) 
defined  the  sense  of  presence  as  having  both  external  and  internal  properties.  Internal 
properties  are  a  person’s  subjective  sense  of  presence  and  external  properties  are  how  the 
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subject  interacted  with  the  VE.  These  researchers  also  have  explored  application  of 
neurolinguistic  programming  to  quantify  the  sense  of  presence.  Hendrix  and  Barfield 
(1995)  measured  the  level  of  presence  within  visual  and  auditory  VEs.  They  found  that 
head  tracking,  larger  field  of  view,  and  stereoscopic  cues  provided  significantly  higher 
sense  of  presence.  Further  investigation  by  Barfield,  Hendrix,  and  Bystrom  (1997) 
support  •  evious  findings  that  head  tracking  improved  the  sense  of  presence. 

Most  of  the  work  in  measuring  the  sense  of  presence  has  focused  on  the  visual 
aspects  of  the  systems.  However,  Gilkey  and  Weisenberger  (1995)  provide  arguments 
that  the  auditory  channel  is  just  as  important  and  may  be  crucial  in  attaining  maximum 
sense  of  presence.  These  researchers  provide  a  corollary  to  deafen  adults,  suggesting  that 
like  experiences  of  the  deaf,  the  sense  of  sound  is  crucial  to  making  one  feel  that  they  are 
in  the  environment.  These  researchers  also  feel  that  more  money  should  be  dedicated  to 
the  creation  of  auditory  systems  for  VEs.  Hendrix  and  Barfield  (1996)  also  found  that 
spacialized  sound  increased  the  sense  of  presence,  but  not  the  sense  of  realism  in  the  VE. 
They  suggest  that  perceived  realism  is  more  influenced  by  changes  that  occur  in  the 
visual  display  rather  than  in  the  auditory  sense.  More  research  needs  to  be  accomplished 
to  develop  standard  metrics  for  measuring  the  sense  of  presence  in  a  VE  and  the  sense  of 
presence  per  task. 

Human  Factors  Research  and  3-D  Displays 

Human  factor  researchers  have  been  interested  in  how  to  design  systems  for 
human  use  and  how  humans  perform  when  using  these  systems  (Sanders  &  McCormick, 
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1993).  Wickens  (1992)  believes  that  3-D  displays  should  be  used  to  represent  3-D 
worlds,  such  as  a  product  design  on  a  computer-aided  design  workstation.  The 
investigator  stresses  that  a  3-D  display  is  more  compatible  with  the  operator’s  mental 
model  of  a  3-D  world  than  is  a  traditional  2-D  display.  While  2-D  representations 
provide  the  user  with  the  necessary  information  to  reconstruct  a  3-D  picture,  2-D 
renderings  require  mental  gymnastics  to  integrate  and  reconstruct  the  picture.  In  an 
earlier  work,  Wickens  (1990)  discussed  two  basic  arguments  for  implementation  of  3-D 
displays:  the  visual  scene  of  a  3-D  world  is  more  intuitive  and  natural  representation  than 
2-D  displays,  and  a  single  integrated  object  reduces  the  need  for  a  mental  integration  of 
two  or  three  representations. 

Sanders  and  McCormick  (1993)  discuss  3-D  displays  in  an  aircraft  environment. 
They  contend  that  an  aircraft  occupies  a  position  in  3-D  space  and  that  pilots  must 
assimilate  and  integrate  information  from  2-D  displays  into  a  coherent  3-D  image  or 
mental  model  of  the  environment.  In  experiments  comparing  pilots’  initiation  of  evasive 
maneuvers  to  avoid  collision,  findings  show  that  pilot  decision  time  with  3-D  displays 
was  3  to  6  seconds  faster  than  with  2-D  displays. 

Wichansky  (1991)  investigated  user  benefits  of  visualization  with  3-D 
stereoscopic  displays.  Data  analysis  from  controlled  human  factor  experiments  showed 
that  user  performance  with  stereoscopic  displays  exceeded  performance  of  those  using  2- 
D  displays  for  various  tasks  (visual  search,  cursor  positioning,  tracking)  and  applications 
(teleoperation,  military  simulation,  and  CAD).  This  researcher  also  conducted  a  study  of 
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a  stereoscopic  display  system  for  computer  workstations  with  20  users  and  third  party 
software  developers,  to  determine  whether  a  3-D  stereo  display  was  perceived  as  better 
than  flat,  2-D  displays.  Results  indicated  that  users  perceived  more  benefits  of  3-D  stereo 
in  applications  such  as  molecular  modeling  and  cell  biology,  which  involved  viewing  of 
complex,  abstract,  amorphous  objects.  In  these  experiments,  users  typically  mentioned 
clearer  visualization  and  better  understanding  of  data,  easier  recognition  of  form  and 
pattern,  and  more  fun  and  excitement  at  work  as  the  chief  benefits  of  stereoscopic 
displays.  Stereoscopic  displays  have  also  been  judged  superior  for  visual  search  and 
interactive  cursor  positioning  tasks,  for  spatial  judgment  tasks,  and  for  communication  of 
design  information  (McWhorter,  Hodges,  &  Rodriguez,  1991).  These  researchers 
conducted  two  experiments  to  rank  different  types  of  display  formats  common  to  CAD 
applications  for  geometric  information  conveyed  and  perceived  realism  of  objects. 
Display  types  tested  were:  wireframe,  wireframe  with  hidden  lines  removed',  shaded 
solid,  orthogonal  multi-view,  stereoscopic  wireframe,  stereoscopic  with  hidden  lines 
removed,  and  stereoscopic  shaded  solid.  The  results  of  the  geometric  information 
experiment  indicated  that  the  orthogonal  multi-view  display  was  judged  inferior  to  both 
the  non-stereo  and  stereo  pictorial  displays  and  that  the  stereo  displays  were  judged 
superior  to  the  non-stereo  displays  in  providing  geometric  information  to  the  subject. 

Human  factor  research  related  to  3-D  displays  show  that  3-D  images  are  better 
suited  for  displaying  complex  data  where  relationships  are  important  (Wichansky,  1991 ; 
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McWhorter,  Hodges,  &  Rodriguez,  1991).  Complexity  similar  to  those  seen  in  CAD 
models. 

Virtual  Environment  Applications  in  Design 

Applications  of  VEs  can  be  found  in  the  airline  industry,  the  automotive  industry, 
and  in  government  organizations. 

Airline  Industry 

In  1989,  Boeing  conducted  a  company-wide  evaluation  of  the  causes  of  there  past 
design  problems  as  experienced  in  the  traditional  design  process.  One  year  later,  based 
on  that  evaluation,  a  corporate  decision  was  made  that  by  1992  all  design  work  would  be 
done  digitally  (Mathes,  1993).  Boeing  began  their  research  in  VEs  by  focusing  on  two 
major  projects. 

The  first  involved  importing  aircraft  CAD  data  into  a  VR  environment  with  the 
goal  of  integrating  it  into  the  design  test,  and  mock-up  process.  Dr.  Mizell,  Director  of 
Boeing  Computer  Services  Research  and  Technology  Organization,  discussed  Boeing’s 
purpose  for  using  virtual  prototypes  indicating  that  electronic  mockups  provided 
company  designers  with  the  opportunity  to  check  interference  and  provided  an 
environment  for  multiple  participants’  involvement  in  the  design  process.  One  of  the 
problems  articulated  by  Dr.  Mitzell  was  the  problem  of  converting  CAD  data  into  VE 
data.  Airplane  virtual  mock-ups  are  5-10  billion  polygons  in  size.  The  enormous  number 
of  polygons  makes  it  computationally  difficult  to  get  the  information  into  memory  and 
difficult  to  display.  These  are  current  limitations  of  the  technologies.  CAD  part  geometry 
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is  orders  of  magnitude  more  complex  than  the  objects  represented  in  virtual  worlds,  and  it 
is  a  tremendous  computational  challenge  to  render  the  images  fast  enough  for  useful 
immersion  interaction  with  the  data.  To  circumvent  the  problem  Boeing  developed 
proprietary  algorithms  to  reduce  the  number  of  polygons  and  improve  rendering  speed 
(Norris,  1995). 

The  second  project  involved  application  of  augmented  reality.  It  provides  the 
ability  to  superimpose  diagrams  or  text  such  as  templates  or  instructions  onto  the  surface 
or  workpiece  (Mitzell,  1994).  Boeing  plans  to  use  this  technology  to  provide  appropriate 
superimposed  information  to  a  factory  worker  for  each  step  in  a  manufacturing  or 
assembly  operation  (Mizell,  1994).  The  goal  is  to  improve  the  productivity  of  workers  by 
getting  them  the  information  they  need  when  they  need  it  (Pimentel  &  Teixeira,  1995). 

Automotive  Industry 

VE  application  has  not  yet  become  an  integral  part  of  new  car  product 
developments,  but  several  engineering  departments  in  the  automotive  industry  are 
looking  at  future  products  through  stereoscopic  display  devices  (Sedgwick,  1993).  The 
automotive  industry  is  investigating  VE  technology  to  identify  its  possible  benefits  and 
areas  of  promising  application.  Mahoney  (1995)  indicated  that  the  automotive  industry  is 
pinning  their  hopes  on  the  creation  of  virtual  prototype  vehicles  that  can  be  manipulated 
in  real  time.  The  investigator  explains  that  the  real  time  aspect  of  vehicle  prototype 
evaluation  provides  tremendous  opportunity  to  develop  the  product  right  the  first  time. 
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Since,  VE  technology  is  still  in  its  infancy,  the  benefits  of  VR  applications  in 
automotive  design  have  been  restricted  to  assessments  of  safety  issues,  evaluation  of 
maintenance  procedures,  and  vehicle  aerodynamics  and  ergonomics.  An  excellent 
example  of  how  VE  can  be  used  for  product  design  evaluation  is  demonstrated  when  an 
engineer  looks  under  the  hood  of  an  automobile  to  examine  a  3-D-engine  compartment. 
Using  a  glove  the  designer  could  then  determine  whether  a  mechanic  could  reach  the 
spark  plugs  or  change  the  oil  filter.  This  demonstrates  the  prevention  of  future 
maintenance  issues. 

The  Federal  government  has  encouraged  the  industrial  use  of  VR  technology.  A 
consortium  of  US  Auto  leaders  and  automotive  suppliers  was  formed  to  establish 
cooperative  efforts  to  speed  research  in  new  technologies  that  could  improve  product 
development  and  manufacturing  (Sedgwick,  1993).  This  consortium,  USCAR,  has  the 
power  to  influence  the  electronics  industry  into  developing  products  that  will  be  faster 
and  provide  increased  computational  power  necessary  for  product  rendering  in  VEs. 

A  proof-of-concept  project  was  developed  by  the  British  Aerospace  Brough 
Laboratory  to  demonstrate  the  use  of  VR  for  product  design.  Researchers  created  the 
interior  of  a  Rover  400  automobile  using  CAD  and  a  graphical  programming  language. 
The  system  consisted  of  a  high  resolution  Eyephone™,  a  Dataglove™,  a  3-D  sound 
module,  and  SGI  computers  for  rendering.  Increased  realism  was  obtained  by  including  a 
real  car  seat  for  the  user  to  sit  on.  Designers  were  able  to  study  the  ergonomics  of  the  car 
interior  accurately  and  change  the  position  of  virtual  parts  when  needed.  Using  this 
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method  it  was  possible  to  redesign  the  whole  vehicle  while  immersed  in  the  simulation. 
Designers  could  for  example,  grab  and  move  the  steering  wheel  assembly  from  the  UK’s 
right  handed  drive  position  to  the  American  left  handed  position  (Burdea  &  Coiffet, 

1994;  Kalawsky,  1993). 

Each  automotive  manufacturer  has  taken  a  different  research  approach  to  VEs  and 
product  design.  However,  the  fundamental  reasons  for  their  research  in  this  area  are  the 
same,  they  all  are  committed  to  replacing  expensive  hardware  prototypes  with  virtual  or 
electronic  ones.  Ford  Motor  Company’s  Vice  President  for  Design,  Jack  Telnack,  is 
quoted  as  saying,  “It  is  my  vision  to  link  seven  worldwide  design  studios  together  with 
computers.  VR  would  allow  Ford  designers  and  engineers  in  England  or  Japan  to  work 
on  projects  together  in  real-time”  (Keebler,  1993).  Strong  visions  from  automotive 
leaders  will  play  an  important  role  in  developing  VE  technology  as  a  viable  tool  for 
product  design.  Today,  General  Motors  (GM)  Corporation  is  using  the  CAVE  approach 
combined  with  GM  proprietary  software,  VisualEyes™.  Ellis  (1997)  explains  that  GM 
felt  that  the  CAVE  system  was  a  natural  way  to  wrap  a  car  around  a  human  using  the 
system  for  design  team  evaluations  of  ergonomic  and  vehicle  operations.  Ford  Motor 
Company  and  Chrysler  Corporation  are  using  traditional  head  mounted  displays  and 
other  stereo  display  devices  (Mahoney,  1995).  Automotive  manufactures  are  beginning  to 
integrate  the  powerwall  technology  for  large-scale  design  evaluations. 
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Government  Organizations 

The  majority  of  the  work  on  simulation  and  VE  technology  is  found  in  the 
Department  of  Defense  (DOD)  and  National  Aeronautics  and  Space  Administration 
(NASA).  A  summary  of  their  work  is  provided. 

Department  of  Defense 

In  the  DOD,  post-cold  war  budget  cuts  have  forced  military  leaders  to  rethink  and 
restructure  the  way  they  train  and  maintain  their  forces.  U.  S.  Army  leaders  are  making 
fundamental  changes  by  placing  a  greater  reliance  on  computer  technology.  Simulation 
is  becoming  key  in  all  aspects  of  operations,  including  research  and  development, 
acquisition,  testing,  evaluation,  and  training.  In  1994  the  then.  Chief  of  Staff  of  the 
Army,  General  Gordon  Sullivan  was  quoted  as  saying  “simulation  helps  us  identify  and 
reduce  risk”  (Kitfield,  1994).  The  inherent  capabilities  of  simulation  offer  a  tremendous 
potential  to  save  time  and  money.  In  the  military,  the  current  trend  is  for  increased 
technological  complexity,  maintaining  the  cutting  of  edge  of  state-of-the-art  technology, 
and  shorter  military  hardware  lifespans.  This  requires  investments  in  technologies  that 
are  flexible,  upgradeable  and  networkable.  The  networking  capability  allows  remote 
simulation  without  having  to  transport  trainees  to  the  simulator  site.  This  is  important 
because  it  is  another  source  of  cost  savings.  VR  provides  this  flexibility  and  networking 
capability  that  ideally  matches  the  needs  of  the  military  (Burdea  &  Coiffet,  1994). 

A  demonstration  of  DOD’s  commitment  is  seen  in  their  current  contracting 
approach.  As  an  example,  General  Dynamics  Electric  Boat  Division  under  contract  with 
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the  U.  S.  Navy  designed  a  nuclear  attack  submarine  using  supercomputers,  advanced 
simulation  software,  and  a  centralized  database.  In  addition,  they  projected  stereoscopic 
images  on  a  special  wall  size  screen,  which  provides  users  with  the  feeling  of  3-D 
immersion.  They  also  placed  a  multi-disciplinary  team  of  designers,  shipyard  operators. 
Navy  representatives,  and  suppliers  in  a  darkened  meeting  room  where  they  could 
simultaneously  discuss  and  evaluate  the  detailed  design  of  submarine  components 
(Ashley,  1995).  The  researcher  further  identified  the  Navy’s  goal  of  reducing  the  design 
period  from  nine  to  five  years,  thus  reducing  costs  significantly. 

NASA 

Space  is  a  conducive  environment  for  exploiting  VE  technology  because  building 
a  physical  prototype  for  iterative  evaluation  and  training  is  not  practically  financially 
feasible.  The  Lockheed  Group,  the  prime  contractor  for  NASA  Hubble  effort,  was  able 
to  examine  a  virtual  prototype  of  the  Hubble  telescope  from  all  angles  and  rehearse  its 
deployment.  Using  VE  technology  during  the  development  and  training  phases  they 
discovered  two  important  flaws  in  the  original  engineering  design  (Hancock,  1993). 
These  flaws  were  discovered  early  enough  for  the  actual  flight  hardware  to  be  fixed.  The 
investigator  postulates  that  if  the  virtual  prototype  had  not  been  available  those  flaws 
would  have  likely  remained  undetected  until  much  later,  after  the  physical  build.  This 
could  have  increased  the  risk  of  the  project  and  might  have  impacted  the  success  of  the 
mission. 


85 


Original  VE  systems  were  developed  to  allow  single  user  immersion.  Networking 
technology  offers  the  capability  to  link  multiple  people  in  an  interactive  virtual 
environment.  The  next  section  provides  information  regarding  multi-user  VE  research. 

Multi-user  Virtual  Environments 

Wang  (1995)  defines  networked  VE  as  an  environment  where  multiple  users  are 
connected  by  a  network  and  can  share  information  with  each  other.  This  networked 
architecture  controls  the  interaction  support,  communication  handling,  and  VE 
knowledge  among  multiple  users.  Much  of  the  fundamental  research  on  multi-user 
virtual  environments  is  found  in  the  computer  science  literature  where  computer 
scientists  are  developing  architectures  and  operating  systems  that  allow  multi-user 
interaction  in  a  VE.  This  research  focuses  on  the  underlying  software  and  hardware 
technology  issues  that  are  essential  to  making  multi-user  VEs  a  reality.  For  instance, 
researchers  are  working  on  improvements  to  networking,  operating  systems,  database 
management,  and  communication  protocols.  Two  representative  systems  are  discussed  to 
demonstrate  the  type  of  research  being  conducted  in  this  area:  Virtual  Environment 
Operating  System  (VEOS)  and  Distributed  Interactive  Virtual  Environment  (DIVE). 

Other  network  systems  include  Bricknet,  which  is  based  on  sharing  object  geometry  and 
behavior  (Singh,  Serra,  Pag,  Wong,  &  Hg,  1995);  and  NPSNET,  which  is  based  on  a 
Defense  Internet  Simulation  protocols  developed  at  Naval  Post  Graduate  School  for 
military  simulation  application  (Broil,  1995). 
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Virtual  Environment  Operating  System  (VEOS)  developed  by  Human  Interface 
Technology  Laboratory  at  the  University  of  Washington  is  an  infrastructure  for  VE 
research  based  on  a  distributed  UNIX  environment  (Bricken  &  Coco,  1994).  VEOS  was 
intended  for  distribution  and  noncommercial  usage.  When  multiple  participants  exist  in  a 
virtual  world,  different  participants  see  different  views.  Each  participant  can  occupy  a 
unique  personalized  world,  sharing  the  public  database  partition  and  not  sharing  private 
database  partitions.  Using  the  VEOS  system  demonstration  of  multi -participant 
interactivity  was  conducted  within  two  projects  called  “Block  World”  and  “Catch." 

Block  World  allowed  four  participants  to  independently  navigate  and  manipulate 
moveable  objects.  Catch  demonstrated  interparticipant  spatial  voice  communication. 
Catch  allowed  individual  participants  to  customize  their  personal  view  of  the  shared 
world  in  terms  of  color,  shape,  scale,  and  texture. 

Carlsson  and  Hagsand  (1993)  developed  DIVE.  Their  research  is  focused  on 
distribution,  collaboration,  interaction  and  multi-user  aspects  of  VE.  The  system  is  based 
on  UNIX,  Internet  protocols,  and  multicast  protocols.  Graphical  objects  called  “body 
icons”  represent  users.  Body  icons  facilitate  awareness  of  ongoing  activities  since  it 
defines  the  position  from  which  the  user  sees  the  world.  Users  can  select  and  grasp 
objects  using  interaction  devices  and  these  selected  objects  are  attached  to  the  body  icon 
for  movement  in  the  VE.  DIVE  also  distributes  messages  to  all  members  of  the  world  and 
these  messages  are  interpreted  by  the  application  so  those  objects  react  autonomously 
according  to  their  pre-assigned  behavior. 


87 


The  combination  of  collaborative  product  design  with  VE  technology  may  play  a 
key  role  in  next  generation  CAD  tools  and  design  process  strategies.  The  next  section 
describes  research  in  the  area  of  collaborative  design  using  virtual  environments.  The 
research  conducted  to  date  has  focused  on  the  initial  developments  of  collaborative 
computer  aided  design  tools  and  studies  to  evaluate  how  multiple  people  will  interact  in  a 
common  design.  The  goal  of  these  studies  is  to  develop  criteria  for  future  collaborative 
virtual  design  environments. 

Collaborative  Virtual  Design  Environments 

Today,  most  CAD  systems  are  limited  in  their  ability  to  allow  designers  to  create 
designs  intuitively.  Before  a  design  can  be  created,  the  designer  must  have  complete 
knowledge  of  all  the  features  they  are  attempting  to  create  (Lansdown,  1994).  Designs 
themselves  are  3-D  in  nature  but  current  CAD  input  devices,  keyboard  and  mouse,  are  2- 
D.  Theses  2-D  input  mechanisms  often  restrict  the  designer  because  objects  must  be 
created  using  points  and  lines.  Even  free  form  must  be  defined  in  terms  of  control  points 
that  bound  it.  This  can  prove  to  be  a  limitation,  especially  during  the  concept  design 
stage  of  the  product  when  the  dimensions  are  not  precisely  known  (Gadh,  1994).  Recent 
research  has  focused  on  applying  VE  technologies  to  the  product  design  process. 

Creating  design  methods  based  on  3-D  tools  in  a  3-D  environment.  This  system  would 
contain  actual  design  information  with  an  added  feature  of  product  creation,  visualization 
and  object  manipulation.  Deering  (1996)  developed  a  VR  sketching  system  called  the 
Holosketch  that  allows  the  construction  and  manipulation  of  3-D  objects  using  3-D  tools. 
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The  Holosketch  system  is  based  on  a  desktop  VR  system  with  head  tracking  stereoscopic 
glasses  and  a  3-D  mouse  and  wand.  The  software  extends  the  2-D  sketch-draw  paradigm 
to  the  third  dimension.  The  system  is  controlled  through  the  use  of  a  3-D  multi-level 
fade-up  pie  menu.  Deering  (1996)  found  that  significant  productivity  gains  are  possible 
over  conventional  2-D  interface  technologies. 

Dani  et  al.  (1994)  developed  a  conceptual  virtual  design  system  (COVIRDS)  that 
creates  an  interactive  3-D  environment  in  which  the  designer  can  use  a  combination  of 
hand  gestures,  voice  input,  and  a  keyboard  to  create  and  view  an  object.  COVIRDS  uses 
VR  hardware  and  software  technology  to  provide  a  designer  with  a  user  interface  that  no 
longer  restricts  the  user  to  a  2-D  screen.  This  development  is  specifically  geared  toward 
the  early  concept  phase  of  development  where  product  dimensions  are  not  precisely 
known.  It  allows  a  more  intuitive  means  for  designer  creativity.  Future  features  will 
include  a  mechanism  to  translate  the  conceptual  design  to  a  format  acceptable  by  CAD 
packages.  The  current  system  was  developed  for  single  user  application. 

Gisi  and  Sacchi  (1994)  developed  a  system  called  Co-CAD.  A  designer  can 
construct  complex  3-D  objects  from  primitive  3-D  objects  such  as  cones,  or  blocks.  This 
system  functions  as  a  single  unit  or  linked  to  form  a  collaborative  design  environment. 
When  operating  in  a  multi-user  capacity  each  site  runs  a  copy  of  Co-CAD  that  shares  the 
semantic  data  of  the  design  to  the  other  systems.  The  researchers  describe  some  features 
that  facilitate  collaborative  interactions:  customizable  local  views,  shared  pointer,  view 
synchronization,  object  ownership  and  access  permissions,  joining  a  design  session 
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already  in  progress,  and  participant  termination  and  failure  robustness.  These  researchers 
found  through  interviews  with  design  engineers  that  this  type  of  tool  would  not  be  useful 
throughout  the  design  phase  instead  they  felt  the  tool  was  more  beneficial  for  intermittent 
consultations  or  design  review  uses. 

Teledesign  (Shu  &  Flowers,  1994),  developed  at  the  Massachusetts  Institute  of 
Technology  Computer  Aided  Design  Laboratory,  is  a  groupware  system  that  allows 
people  to  modify  a  common  design.  The  system  was  implemented  to  study  user  interface 
issues.  Teledesign  supports  real  time,  physically  dispersed  or  face  to  face  meetings  of 
several  people  using  a  replicated  architecture.  This  experimental  architecture  requires 
each  linked  site  to  run  individual  software  applications.  It  allows  sequential  access  where 
only  one  designer  can  enter  edits  at  a  time  and  simultaneous  access  where  all  the 
designers  can  edit  at  the  same  time.  Two  experiments  were  conducted  using  five  groups 
of  two  subjects  each.  In  the  first  experiment,  the  primary  goal  was  to  determine  the 
effects  of  different  modes  of  edit  access  or  floor  passing  schemes  on  the  performance  of 
different  types  of  collaborative  design  tasks.  Two  modes  were  investigated:  simultaneous 
editing  and  sequential  editing.  It  was  revealed  that  a  simultaneous  mode  of  edit  access 
was  preferred  over  a  forced  turn-taking  mode.  The  second  experiment  was  designed  to 
evaluate  performance  of  a  software  system  expert-novice  team,  where  the  expert 
demonstrated  the  available  functions  and  system  features.  It  was  found  that  the  use  of  a 
telepointer  was  critical  during  the  teaching/instructional  phases.  It  was  also  found  that 
allowing  designers  to  have  independent  point  of  views  optimized  parallel  activity  when 


independent  tasks  were  performed.  Also,  the  need  for  a  3-D  pointer  and  knowledge  of  the 
other  designer’s  point  of  view  was  established. 

The  Institute  of  Simulation  and  Training  at  the  University  of  Central  Florida 
developed  a  prototype  system  called  Polyshop  which  utilized  a  unique  two  handed 
interface  and  an  immersive  stereoscopic  display  system  that  allowed  a  modeler  the  ability 
to  manipulate  data  in  a  true  3-D  perspective.  The  two-handed  direct  manipulation 
interface  described  by  Mapes  and  Moshell  (1995)  is  based  on  real  world  object 
manipulation.  As  in  the  real  world,  it  allows  the  user  to  use  head  rotation  to  easily  view 
the  object  and  allowed  the  user  to  grasp  objects  for  manipulation.  The  prototype  system 
featured  simple  operations  of  object  rotation,  translation,  scaling,  alignment,  and  gluing 
more  than  one  object  together.  The  system  was  developed  as  a  rapid  construction  model 
for  urban  combat  mission  training  (Blau,  Mapes,  &  Moshell,  1993).  This  prototype 
further  developed  and  commercialized  by  Multigen,  Inc.,  a  product  called  SmartScene™ 
(Multigen,  1998).  This  product  allows  for  real-time  3-D-scene  assembly.  Users  are 
immersed  in  a  3-D  visual  workspace,  and  within  this  virtual  workspace  using  a  two- 
handed  interface  and  Smart  Model  Time  Behaviors  can  create  3-D  world  objects.  The 
two-handed  interface  incorporates  natural  hand  gestures  to  grab  and  manipulate  models 
with  two  hands.  The  Model  Time  Behavior  allows  users  to  quickly  snap  objects  together, 
position  them,  and  scale  them.  This  system  is  targeting  the  visual  simulation,  urban 
planning,  entertainment,  and  mechanical  assembly  tasks  and  markets. 
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The  literature  clearly  demonstrates  that  new  emerging  VE  technologies  have  the 
potential  to  enhance  the  design  process  by  offering  a  more  natural  mechanism  for  design 
teams  to  assemble  and  evaluate  future  product  concepts. 

Summary 

The  product  development  literature  clearly  identifies  a  need  for  organizations  to 
strive  towards  decreasing  time  to  market,  to  improving  product  quality  and  to  creating 
customer  interfaces  as  integral  parts  of  the  design  process.  These  basic  requirements  are 
necessary  for  organizations  to  remain  competitive. 

The  literature  surrounding  the  concept  design  process  indicates  that  design  is  a 
creative  process,  where  visualization  promotes  identification  of  design  problems  by 
providing  humans  with  feedback  to  analyze  concepts  and  this  potentially  can  help  to 
achieve  improved  product  design  efficiency  and  effectiveness.  Visualization  can  be 
achieved  through  pictorial  representations,  physical  mockups  or  virtual  replicas  of  a  real 
object. 

Product  design  reviews  are  critical  milestones  in  product  development  efforts. 
Individuals  are  assembled  to  evaluate  the  progress  of  a  design.  Team  design  reviews 
have  been  beneficial  for  improving  product  designs  because  control  and  decisions  are 
placed  in  the  hands  of  functional  experts,  participants  take  stake  in  the  outcome,  and 
teams  generate  a  synergy  of  expertise  that  assists  in  problem  solving. 

Product  design  relies  on  the  activity  of  teams  of  individuals  with  varying 
backgrounds  and  experience.  Concurrent  engineering  methodologies  have  been 
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successful  in  improving  product  design  cycle  time  and  product  quality  by  allowing  multi¬ 
functional  teams  to  interact  and  share  information  giving  these  teams  the  capability  to 
make  more  informed,  better  decisions.  Collaboration  or  group  work  has  also  been 
investigated  within  the  field  of  computer  supported  cooperative  work  where  researchers 
are  interested  in  developing  a  greater  understanding  for  automation  of  group  activities 
and  tool  interaction.  The  literature  in  this  area  demonstrates  the  initial  kindling  of 
researchers  interested  in  developing  collaborative  product  design  tools. 

Product  design  is  in  a  state  of  evolution,  depending  heavily  on  improvements  in 
computer  technology.  Transitions  occurred  from  traditional  2-D  hand  drawings  to  3-D 
solid  model  CAD  that  has  automated  the  performance  of  individual  designers,  cutting 
design  times  dramatically.  This  reduction  of  time  and  cost  is  directly  attributed  to  the 
fact  that  the  designer  is  provided  with  a  set  of  tools  that  enhance  product  design  activity. 

A  new  emerging  technology,  VEs  hold  promise  for  product  design  improvement. 
VE  offers  a  potential  to  cut  product  design  time  and  to  improve  product  quality  because 
VE  has  the  capability  to  provide  a  sense  of  reality  to  a  product  design  and  to  allow 
multiple  people  to  interact  in  the  same  virtual  design  environment  without  ever  building  a 
physical  prototype. 

This  review  of  relevant  literature  demonstrates  that  the  concepts  and  theories 
which  compromise  and  mold  the  intriguing  problem  of  integrating  the  requirements  of 
VE  visualization  tools  and  automation  into  a  multi-functional  collaborative  product 
design  environment  is  still  in  an  eariy  stage  of  research.  Researchers  have  only  begun  to 
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understand  the  complexity  of  how  to  use  commercial  visual  display  devices,  how  to 
develop  better  design  tools,  and  how  to  use  them  in  a  more  efficient  and  effective 
manner. 
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CHAPTER  3 


METHODS  AND  PROCEDURE 

This  study  was  conducted  at  the  U.  S.  Army  Tank-automotive  and  Armaments 
Command  (TACOM)  in  Warren.  Michigan.  Research  data  were  collected  over  a  four-day 
period  to  empirically  examine  the  effects  of  3-D  stereoscopic  visual  display  technologies 
on  the  CDR  process.  Test  subjects  representing  several  functional  areas  were  grouped 
into  three  person  design  teams.  These  teams  conducted  a  design  review  task  using  each  of 
the  four  visual  display  systems  being  investigated.  This  chapter  details  the  research 
design,  data  collection  procedures,  and  data  analysis  methodology. 

Research  Design 

The  purpose  of  this  study  was  to  evaluate,  through  comparative  empirical  testing 
and  data  analysis,  how  several  commercially-available  virtual  3-D  visual  display  systems 
contribute  to  cross-functional  team  collaboration  in  a  product  design  review 
environment.  These  computer  visual  display  technologies,  often  referred  to  as  output 
devices  for  VE  systems,  allow  users  to  perceive  3-D  objects  in  virtual  space.  This  study 
focused  on  an  evaluation  of  several  visual  display  technologies  that  are  currently  being 
used  and  evaluated  in  various  public  and  private  sector  product  development 
organizations.  Display  systems  tested  were  a  Helmet  Mounted  Display  (HMD), 

Binocular  Omni-Orientation  Monitor  (BOOM),  Stereoscopic  glasses  and  monitor  (with 
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IR  emitter  and  sensor),  and  for  comparison  a  traditional  monoscopic  CRT  monitor. 
Figure  3  illustrates  the  visual  display  technologies  used  in  this  study. 


Figure  3.  Commercial  Visual  Display  Systems 
Visual  display  systems  are  often  categorized  by  how  the  observer  uses  his  or  her 
eyes  to  create  a  visual  image.  Monoscopic  systems  display  the  same  2-D  image  to  both 
eyes.  In  contrast,  stereoscopic  systems  present  a  different  image  to  each  eye,  creating  the 
effect  of  depth.  Table  1  associates  the  visual  display  systems  used  in  this  study  with  the 
type  of  display  system.  As  shown,  three  of  the  test  visual  display  systems  were 
stereoscopic  devices  and  for  comparison  one  monoscopic  system  was  used. 
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Table  1. 


Visual  Display  System  Categorization 


Helmet  Mounted  Display 

System  (HMD) 

Stereoscopic 

Binocular  Omni- 
Orientation  Monitor 
(BOOM) 

Stereoscopic 

Stereoscopic  Glasses  and  Monitor 
(IR  Emitter/Sensor) 

Stereoscopic 

Traditional  CRT  Monitor 

Monoscopic 

It  was  hypothesized  for  this  study  that  the  use  of  VE  technologies  and  their 
capability  would  allow  users  to  visualize  and  interact  in  virtual  concept  designs  and 
conduct  CDRs  more  effectively  and  efficiently,  and  their  use  would  improve  the  design 
process  and  the  inherent  qualities  of  end  products.  These  improvements  can  be 
associated  with  the  capabilities  that  visualization  technologies  offer.  It  was  also 
hypothesized  that  these  technologies  may  have  the  potential  to  provide  product  design 
teams  with  a  common  platform  to  evaluate  early  conceptual  designs  and  may  assist  in  the 
resolution  of  design  problems.  However,  no  existing  empirical  data  supports  these 
hypotheses.  Only  broad  claims  demonstrating  the  value  of  the  technologies  have  been 
made  (Ellis,  1997).  The  research  design  method  used  in  this  study  served  to  provide 
insight  into  these  broad  claims.  Five  questions  were  developed  to  address  the  impact  of 
3-D  visual  display  technologies  on  the  CDR  process.  They  are: 
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1 .  Do  design  teams  detect  and  identify  more  design  problems  and  design  errors 
when  using  3-D  visualization  tools? 

2.  Do  design  teams  detect  and  identify  design  problems  and  errors  more  quickly 
when  using  3-D  visualization  tools? 

3.  Do  design  teams  resolve  design  problems  more  quickly  when  using  3-D 
visualization  tools? 

4.  What  is  the  perceived  impact  on  the  design  process,  the  quality  of  the  design 
process,  and  the  physiological  state  of  the  design  team  members  after  using 
3-D  visualization  tools? 

5.  When  comparing  the  four  visual  display  technologies,  what  are  preferences  of 
their  usefulness,  difficulty,  practicality,  stimulation  for  team  interactivity,  and 
development  of  team  consensus? 

Subjects 

CDRs  are  periodically  conducted  throughout  the  conceptual  design  and  product 
development  process.  Design  teams  are  formed  using  volunteer  representatives  from 
several  functional  areas  within  an  organization.  These  teams  consist  of  members  with 
differing  educational  backgrounds  and  functional  responsibilities. 

Participants  for  the  study  were  personnel  from  the  TACOM  center  in  Warren. 
Michigan.  TACOM  is  a  Department  of  the  Army  organization  with  4,800  employees 
whose  responsibilities  encompass  the  design,  development,  and  procurement  of  military 
ground  vehicle  systems.  The  sample  population  consisted  of  12  male  volunteers  from  the 
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following  operational  organizations:  new  equipment  training,  human  factors,  logistics 
and  maintenance,  configuration  management,  quality  assurance,  and  design. 

Participants  were  tested  for  color  deficiency  and  visual  acuity  prior  to  the 
experimentation  using  a  standard  Ishihara  Color  Deficiency  test  (Ishihara,  1996)  and  a 
Snelling  Chart  Examination.  Participants  that  tested  positive  for  color  deficiency  were 
eliminated  from  the  subject  pool.  Design  teams  were  purposefully  created  with  similar 
backgrounds  and  characteristics  based  on  an  aggregation  of  their  work  experience, 
computer  usage,  education,  and  design  review  experience.  Table  2  indicates  the 
demographics  of  the  test  population.  The  table  summarizes  test  participant  background 
data  and  design  team  composition.  As  shown,  each  design  team  consisted  of  one 
designer  and  two  non -designers. 

Experimental  Design 

The  experimental  design  used  in  this  study  is  a  4x4  Graeco-Latin  Square  as 
described  by  Box,  Hunter,  and  Hunter  (1978).  This  experimental  design  permits  the 
study  of  four  treatments  (technologies)  simultaneously  with  three  different  blocking 
factors  (teams,  TFTS  subassemblies,  and  experiment  order).  A  matrix  showing  the 
relationships  between  visual  display  technologies  (A-D),  teams  (T),  TFTS  subassemblies 
(1-4),  and  experiment  order  (E)  is  depicted  in  Figure  4. 
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Table  2. 


Summary  VE  Participant  Background  Data 


Test  Participants  and  Their  Backgrounds 

Subject 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

ii 

12 

Team 

T, 

T, 

t3 

t4 

Type 

D 

N 

N 

D 

N 

N 

D 

N 

N 

D 

N 

N 

Organization 

D 

NT 

CM 

D 

M 

NT 

D 

HF 

CM 

D 

QA 

L 

Education 

3 

2 

2 

3 

4 

2 

3 

3 

3 

4 

3 

1 

Work 

Experience  in 
Field 

5 

6 

6 

5 

4 

5 

3 

6 

4 

4 

6 

5 

Number  of 
Computer 
Systems 
Experienced 

6 

3 

6 

6 

5 

1 

3 

1 

5 

3 

2 

4 

Number  of 

3-D 

Visualization 

Tools 

Experienced 

4 

3 

1 

3 

1 

0 

4 

0 

0 

4 

0 

1 

Number  of 

Design 

Reviews 

6 

0 

0 

6 

0 

0 

2 

6 

0 

2 

0 

3 

! 

I 

l 

Notes: 

Type:  D  =  Designer;  N  =  Non-designer 

Organization:  D  =  Design;  NT  =  New  Equipment  Training;  CM  =  Configuration 
Management;  HF  =  Human  Factors;  QA  =  Quality  Assurance;  L  =  Logistics 

Education:  1  =  High  School;  2  =  Associates  Degree;  3  =  Bachelors  Degree;  4  =  Masters 
Degree  and  Post  Graduate 

Work  Experience:  1  =  less  than  one  year;  2  =  1  up  to  3  years;  3  =  3  up  to  5  years;  4  =  5 
up  to  10  years;  6  =  10  up  to  15  years 
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Teams 


c 

V 


V 

a. 

x 


T,  T2  T,  T4 

E, 

A1 

B2 

C3 

D4 

E, 

B3 

A4 

D1 

C2 

^3 

C4 

D3 

A2 

B1 

e4 

D2 

Cl 

B4 

A3 

Visual  Display  Technologies 

A  =Helmet  Mounted  Display  System  (HMD) 

B  =  Binocular  Omni-Orientation  Monitor  (BOOM) 
C  =  Stereoscopic  Glasses  and  Monitor 
D  =  3-D  Monoscopic  CRT  Monitor 


1TTS  Subassemblies 

1  =  Fuel  Transfer 

2  =  Fuel  Container 

3  =  Towing 

4  =  Suspension 


Figure  4.  Experimental  Design:  A  4x4  Graeco-Latin  Square 
The  Graeco-Latin  Square  was  selected  for  its  appropriateness  to  the  research 
hypotheses  proposed  in  this  study.  The  treatments  that  are  to  be  compared  are  four  visual 
display  technologies.  These  technologies  are  referenced  as  A,  B,  C,  and  D  and  are 
defined  as  a  HMD,  BOOM,  stereoscopic  glasses  and  monitor,  and  traditional  monoscopic 
CRT  monitor,  respectively.  The  three  blocking  factors  were  the  four  design  teams  (T i. 
T2.T3.  T4),  the  four  different  TFTS  subassemblies  (1,  2,  3,  4),  and  the  four  experimental 


orders  (E|.  E2.  E3.  E4). 


The  Graeco-Latin  Square  Model  protects  against  the  possible  confounding  effects 
that  jeopardize  the  internal  validity  of  an  experiment.  Campbell  and  Stanley  (1966) 
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defined  seven  threats  to  the  internal  validity  of  an  experiment  as  follows: 

1.  History:  Any  events  occurring  between  the  first  and  second  measurement  of 
the  experiment,  in  addition  to  the  experimental  variable. 

2.  Maturation:  Processes  of  the  subjects  operating  as  a  function  of  the  passage  of 
time,  such  as  growing  older,  growing  hungrier,  and  growing  more  tired. 

3.  Testing:  The  effects  of  taking  a  test  upon  scores  of  a  second  test. 

4.  Instrumentation:  When  changes  in  the  calibration  of  an  evaluation  instrument 
or  changes  in  the  observers  and  administrators  of  a  test  may  produce  changes 
in  the  obtained  measurements  and  scores. 

5.  Statistical  Regression:  A  circumstance  occurring  where  groups  have  been 
selected  on  the  basis  of  their  extreme  scores. 

6.  Biases  resulting  in  differential  selection  of  respondents  for  the  comparison 
groups. 

7.  Experimental  Mortality:  The  loss  of  subjects  from  the  comparison  groups, 
which  are  potential  threats  to  the  internal  validity  of  an  experiment,  (p.  5) 

In  this  study,  the  duration  of  the  experiment  was  one  week,  with  each  design  team 
completing  their  experimentation  in  one  day.  The  short  time  in  between  each  treatment 
precludes  possible  effects  of  history  and  maturation.  In  addition,  because  purposeful 
assignment  of  design  team  members  and  randomization  of  factor  assignment  controlled 
initial  differences  between  design  teams,  threats  such  as  testing  and  statistical  regression, 
if  they  occurred  at  all,  affected  design  teams  equally.  A  single  independent  test  observer 
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collected  data  for  the  experiment  and  subjective  data  was  collected  on  printed 
questionnaires.  These  controlled  for  changes  in  instrumentation.  The  treatment  period  of 
the  study  was  extremely  short,  a  one  hour  duration.  The  short  time  span  eliminated  any 
effect  of  differential  mortality  between  treatments.  In  summary,  seven  threats  to  internal 
validity  of  the  study  were  controlled. 

External  validity  refers  to  the  generalization  of  results  of  an  experiment  to  other 
populations  and/or  settings.  The  study  involved  the  task  of  concept  design  review  and 
evaluation.  Caution  is  suggested  when  generalizing  findings  from  this  research  to  other 
tasks  or  settings.  Details  of  the  TFTS  concept  used  for  this  study  and  operational 
requirements  were  provided  to  form  a  basis  for  estimating  the  degree  to  which  research 
findings  were  likely  to  apply  to  other  tasks.  In  addition,  the  use  of  U.  S.  Army  personnel 
assembled  into  teams  suggested  caution  in  generalizing  findings  from  this  research  to 
individual  performances  or  to  other  team  composition.  Information  regarding  team 
composition,  such  as  organization,  education,  work  experience,  computer  usage,  and 
design  review  experience  of  the  research  sample  was  provided  to  form  a  basis  for 
estimating  the  degree  to  which  research  findings  were  likely  to  apply  to  other 
populations. 


Tracked  Fuel  Trailer  System  (TFTS)  Model  Development 

The  TFTS  was  an  exploratory  idea  being  investigated  by  the  U.  S.  Army 
TACOM.  The  TFTS  was  based  on  novel  operational  requirements  for  a  trailer  system 
that  could  carry  a  600-gallon  fuel  pod  and  that  would  continuously  provide  fuel  to  an 
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Abrams  Ml  Main  Battle  Tank  while  in  tow.  Appendix  B  provides  the  operational 
requirements  for  the  system.  Based  on  the  operational  requirements  one  concept  of  the 
TFTS  was  designed,  modeled,  and  translated  into  a  virtual  software  environment.  Figure 
5  pictures  the  conceptual  model  of  the  Abrams  Ml  Main  Battle  Tank  and  TFTS  used  for 
this  study. 


Figure  5.  Abrams  Ml  Main  Battle  Tank  and  TFTS 


The  TFTS  design  was  separated  into  four  major  subassemblies.  These  are  the  fuel 
transfer,  fuel  container,  suspension,  and  towing  subsystems.  Figure  6  identifies  the  four 
subassemblies  used  in  this  study. 

The  four  subassemblies  of  the  TFTS  contained  design  problems  and  design  issues 
with  similar  levels  of  complexity  based  on  an  evaluation  by  a  senior-level  designer. 

These  problems  and  issues  are  related  to  mechanical,  ergonomic,  maintenance,  and 
operational  functions. 
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Figure  6.  TFTS  Subassemblies 

The  model  of  the  TFTS  was  designed  using  commercial  Intergraph  CAD 
software.  The  TFTS  CAD  file  was  then  translated  using  commercial  DIVISION,  Inc.  VE 
software.  This  software  uses  dVS  as  its  virtual  reality  operating  system.  The  dVS 
operating  environment  is  a  flexible  environment  for  the  development  of  immersive  and 
desktop  VR  applications.  In  addition,  DIVISION  uses  dVISE  as  a  VR  interactive 
authoring  tool.  This  tool  allows  for  the  creation  and  experiencing  of  complex  virtual 
environments.  The  software  is  capable  of  interfacing  with  the  four  visual  display 
configurations  being  investigated  in  this  study. 
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Treatment  Conditions 


To  provide  four  treatment  visualization  technologies  necessary  for  this  study,  four 
different  hardware  visual  display  configurations  were  set-up: 

1.  HMD 

2.  BOOM 

3.  Stereoscopic  Glasses  and  Monitor 

4.  Monoscopic  CRT  Monitor 

Helmet  Mounted  Display  (HMD) 

The  HMD  hardware  visual  peripheral  device  used  for  this  study  was  a  Virtual 
Research  Systems,  Inc.  VR4™  Helmet  Mounted  Display  system.  The  set-up  also 
included:  Silicon  Graphics  Onyx  RE2  computer  system,  21 -inch  CRT  monitor.  Virtual 
Research  Systems  controller  box,  and  a  Division,  Inc.  3-D  pointer  device.  Figure  7 
provides  a  schematic  of  the  HMD  configuration  used  in  the  experiment.  As  illustrated, 
the  test  participants  used  a  single  HMD  with  electromagnetic  head  tracking,  a  3-D  Mouse 
Pointer  for  navigation,  and  a  monitor  system  for  team  viewing. 

During  testing  participants  individually  shared  the  HMD  resource.  While  the  test 
operator  used  the  stereoscopic  HMD  device,  the  remaining  team  members  viewed  the 
operators  perspective  on  a  computer  monitor.  Test  participants  took  turns  using  the 
HMD  and  watching  the  monitor. 
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Figure  7.  HMD  Hardware  Configuration 


3-D  Pointer  Navigation  for  HMD 

Figure  8  diagrams  the  3-D  pointer  device  that  was  used  to  navigate  in  the  virtual 
world.  As  shown  in  the  top  view,  the  left  button  is  used  to  move  forward,  the  right  button 
is  used  to  move  backward,  and  if  depressed  the  center  button  brings  up  an  interactive  tool 
bar  menu  that  allows  immersed  users  the  capability  to  alter  the  virtual  model.  The  center 
button  was  deactivated  and  not  used  for  this  experiment.  The  side  view  shows  the  handle 
grip  of  the  3-D  pointer  device.  The  top  button  can  be  depress  using  the  index  finger,  and 
has  the  function  of  selecting.  The  bottom  button  is  only  used  in  combination  with  the 
forward  or  reverse  button,  in  order  to  set  the  viewers  height  in  the  virtual  world.  For 
instance,  if  the  viewer  wants  to  see  the  virtual  model  from  the  top  perspective,  system 
users  would  have  to  physically  look  up  with  their  HMD  on,  depress  both  the  forward  and 
set  height  buttons.  This  would  move  the  viewer’s  perspective  to  the  top  of  the  model. 
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Figure  8.  3-D  Pointer  Functionality 


Binocular  Omni-Orientation  Monitor  (BOOM) 

The  BOOM  hardware  visual  peripheral  device  used  for  this  study  was  a 
Fakespace,  Inc.  BOOM3C™  model.  Figure  9  illustrates  the  BOOM  configuration  for  the 
experiment.  As  shown,  the  system  consists  of  a  single  BOOM  device  linked  to  a  system 
21 -inch  monitor  for  concurrent  viewing.  The  set-up  also  included:  Silicon  Graphics  Onyx 
RE2  computer  system  and  a  Fakespace  interface  box.  Inherent  in  the  BOOM  design  are 
six  encoders  that  track  the  user  position  in  the  virtual  world. 


Figure  9.  BOOM  Hardware  Configuration 
Test  participants  individually  operated  the  stereoscopic  BOOM  resource.  While 
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the  test  operator  used  the  BOOM  device,  the  remaining  team  members  viewed  the 
operators  perspective  on  the  system  monitor.  Test  participant  took  turns  using  the 
BOOM. 

Navigation  for  BOOM 

Figure  10  illustrates  the  BOOM  navigational  device  that  is  used  to  move  in  the 
virtual  world.  As  illustrated,  the  functionality  of  the  right  button  is  acceleration,  the  left 
button  is  used  for  de-acceleration,  and  simultaneously  depressing  both  buttons  stops  any 
movement. 


Stereoscopic  Glasses  and  Monitor 

The  stereoscopic  glasses  visual  peripheral  device  used  for  this  study  was  a 
StereoGraphics,  Inc.,  CrystalEyes™  product.  CrystalEyes  hardware  includes  eyeware 
and  an  infrared  (IR)  emitter.  Figure  1 1  provides  a  schematic  of  the  stereoscopic  glasses 
configuration  for  the  experimentation.  As  illustrated,  the  experimentation  used  a  Silicon 
Graphics  Onyx  RE2  computer  system,  21-inch  computer  monitor,  stereoscopic  glasses, 
electromagnetic  emitter,  and  mouse. 
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Figure  1 1.  Stereoscopic  Glasses  and  Monitor  Hardware  Configuration 


The  stereoscopic  glasses  configuration  allowed  all  test  participants  to  view  the 
virtual  world  simultaneously.  Each  participant  wore  a  pair  of  stereoscopic  glasses  that 
created  the  3-D  stereoscopic  visual  perception.  However,  the  mouse  device  allowed  only 
one  person  to  navigate  the  computer-based  model  at  a  time.  Test  participants  shared  the 
mouse  resource.  While  one  person  navigated  using  the  mouse  device,  the  other  team 
participants  viewed  the  operator’s  perspective  through  a  computer  monitor  and 
stereoscopic  glasses. 

Mouse  Navigation  for  Stereoscopic  Glasses  and  Monitor 

Figure  12  illustrates  the  mouse  device  that  is  used  to  navigate  in  the  virtual  world. 
As  shown  in  the  top  view,  the  center  button  is  used  to  move  right,  left,  up,  or  down. 
Depressing  both  the  center  and  right  buttons  creates  a  panning  effect  that  allows  the  user 
to  freely  move  in  any  direction.  The  left  button  is  used  for  selecting  objects.  For  this 
experiment,  no  object  selection  was  required. 
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Monoscopic  CRT  Monitor 

Figure  13  provides  a  schematic  of  the  monoscopic  configuration  for  the 
experiment.  As  illustrated,  the  experiment  used  a  Silicon  Graphics  Onyx  RE2  computer 
system,  21 -inch  computer  monitor  and  a  mouse.  By  definition  the  monoscopic 
configuration  does  not  provide  the  users  with  any  depth  perception,  analogous  to 
traditional  desktop  PC  computer  systems. 


Figure  13.  Monoscopic  CRT  Monitor  Hardware  Configuration 
Similar  to  the  Stereoscopic  glasses  and  monitor  configuration,  the  monoscopic 
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configuration  allowed  all  test  participants  to  share  the  computer  monitor  and 
simultaneously  view  the  virtual  model.  However,  the  mouse  device  only  allowed  one 
person  at  a  time  to  navigate  the  computer-based  model  at  a  time.  Test  participants  shared 
the  mouse  resource.  While  one  person,  navigated  and  used  the  mouse  device,  the  other 
team  members  viewed  the  operator’s  perspective  by  looking  at  the  computer  monitor. 

Mouse  Navigation  for  Monoscopic  CRT  Monitor  Configuration 

Operation  and  navigation  in  the  monoscopic  setup  is  identical  to  the  stereoscopic 
glasses  and  monitor  configuration.  For  more  detail  refer  to  Figure  12  and  the  description 
paragraph  associated  with  navigation  for  stereoscopic  glasses  configuration. 

A  summarized  listing  of  some  of  technical  and  general  specifications  of  the  four 
visual  display  systems  tested  in  this  study  is  provided  in  Table  3.  FOV  is  defined  as  the 
part  of  space  you  can  see  immediately,  without  moving  your  head  or  your  eyes.  In  terms 
of  a  visual  display,  the  FOV  is  the  visual  angle  subtended  by  the  display  (Stuart,  1996). 

In  each  configuration,  a  separate  21 -inch  CRT  monitor  was  used  for  team  participant 
viewing.  As  shown,  the  cost  of  the  visual  displays  range  from  $800  per  individual 
Crystal  Eye  glasses  to  over  $100,000  for  a  BOOM  system.  The  systems  were 
implemented  using  a  SGI  Onyx  Reality  Engine  2  computer  with  graphics  capability 
costing  approximately  $350,000.  The  cost  of  this  system  may  be  reduced  as  newer,  SGI 
Infinite  Reality  machines  are  introduced  into  the  marketplace. 

Data  Collection 

Experimental  tasks,  data  collection  procedures,  and  instrumentation  were  required 
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for  investigation  of  four  visual  display  technologies  on  their  contribution  to  the  CDR 
process.  This  study  employed  several  evaluation  approaches  documented  in  human- 
computer  interface  and  usability  literature.  These  approaches  were:  empirical  evaluation 
of  the  task,  Likert-like  questionnaires,  human  interface  comparisons,  and  open-ended 
questions  (Smart,  1996;  Nielsen,  1993;  Bardsley  &  Sexton,  1997).  The  later  two  methods 
were  designed  to  gather  supplemental  data  regarding  the  use  of  the  four  visual  display 
technologies.  This  researcher  developed  a  test  plan  summary  that  outlines  the  research 
design  and  methodology  for  completion  of  this  study.  A  copy  of  the  test  plan  summary 
is  contained  in  Appendix  C. 


Table  3. 

Test  Visual  Display  Systems  Specifications 


Company 

Model 

FOV 

(degrees) 

Type  of 
Optics 

Resolution 

Weight 

(Ounces) 

Cost 

1998  ($) 

Virtual  Research 
Systems,  Inc. 

VR4 

60 

Dual  LCD 

640x480 

33 

6,900 

Fakespace,  Inc. 

BOOM 

3C 

45 

CRT 

640x480 

**0 

1 10.700 

Stereographies 

Inc. 

Crystal 

Eyes 

*70H 

MOV 

Liquid 

Crystal 

Shutter 

1280x1024 

3.3 

Glasses  795 

Emitter  200 

Silicon  Graphics, 
Inc. 

CRT 

Monitor 

Full 

Monitor 

None 

1280x1024 

0 

0 

Notes:  *  CrystalEyes  FOV  is  the  infrared  angle  of  view  of  the  emitter.  H=horizontal  and 
V=vertical 

**  BOOM  weight  is  interpreted  as  the  weight  a  user  would  sustain  when  using  the 
system. 

Each  test  system  configuration  included  a  21 -inch  SGI  CRT  monitor  which  costs 
$3,000  and  a  SGI  Onyx  RE2  which  costs  approximately  $350,000.  These  costs 
reflect  government  pricing. 
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Experimental  Tasks 


Traditionally,  design  reviews  are  conducted  periodically  during  the  product 
development  and  design  process.  The  intent  of  these  reviews  is  to  identify  problems, 
formulate  solutions,  and  develop  a  consensus  among  design  review  participants. 
Emulating  this  environment,  the  experimental  task  consisted  of  having  multi-functional 
design  teams  identify  and  solve  design  problems  with  the  assistance  of  four  visual  display 
systems. 


Procedures 

Dlustrated  in  Figure  14  is  the  methodology  used  in  conducting  this  study.  The 
procedures  are  separated  into  seven  sections:  VE  Assessment  Survey  One,  pre-test 
orientation,  test  protocol,  technology  familiarization,  experiment  and  data  collection,  VE 
Assessment  Survey  Two,  and  VE  Assessment  Survey  Three. 


VE  Assessment  One 
(Participant  Background) 

Pre-Test 

Orientation 

Test  Protocol 

i 

i 

Technology 

Familiarization 

◄ 

V 

Experiment  and 
Data  Collection 

VE  Assessment  Two 
(Individual  Technology) 


VE  Assessment  Three 


Figure  14.  Design  Methodology  and  Procedures 
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Prior  to  formal  experimentation  at  the  U.  S.  Army  TACOM,  a  pilot  test  served  to 
validate  the  experiment  and  experimental  procedures  used.  One  team  consisting  of  three 
members  volunteered  to  participate  in  the  pilot  test.  Using  the  same  experiment 
procedures,  hardware  configurations,  TFTS  model,  and  experiment  location  as  the  final 
experimentation  pilot  subjects  conducted  CDRs  using  the  four  test  visual  display  systems. 
The  pilot  design  team  used  each  of  the  four  visual  display  technologies  to  evaluate  each 
of  the  four  TFTS  subassemblies.  Result  from  the  pilot  study  identified  potential 
experimental  set-up  deficiencies,  enabling  improved  experimental  organization. 

Difficulty  in  taking  time  measurements  for  team  performance  was  identified  and 
corrected.  A  change  in  the  design  protocol  was  made  requiring  teams  to  conduct  their 
CDRs  in  a  sequential  manner  by  identifying  and  resolving  one  design  problem  at  a  time 
before  moving  to  the  next.  This  eliminated  the  data  collection  problem  with  regard  to  the 
sequencing  and  timing  events.  The  pilot  test  also  provided  confirmation  of  the  structure 
of  the  questionnaires  and  identified  areas  for  improvement  on  the  data  collection  forms. 

VE  Assessment  Survey  One:  Participant  Background 

A  VE  Assessment  Survey  One  questionnaire  was  designed  by  the  researcher  to 
obtain  test  participant  background  information.  A  copy  of  this  survey  is  contained  in 
Appendix  D.  The  questionnaire-included  sections  on  educational  background, 
professional  work  experience,  level  of  computer  usage,  and  design  review  experience. 
Questions  regarding  participant’s  computer  experience  were  based  on  a  Questionnaire  for 
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User  Interface  Satisfaction  (QUIS)  created  at  the  Human-Computer  Interaction 
Laboratory  at  the  University  of  Maryland  (Chin,  Diehl,  &  Norman,  1988).  Participants 
completed  VE  Assessment  Survey  One  prior  to  the  start  of  experimentation. 
Questionnaire  data  were  used  to  assign  participants  to  four  design  teams  to  achieve  team 
balance  with  regard  to  their  work  experience,  computer  experience,  education,  and  design 
review  experience.  A  total  of  three  test  participants  composed  each  design  team.  Each 
design  team  consisted  of  one  designer  and  two  non-designer  functional  members. 

Pre-test  Orientation 

A  pre-test  orientation  was  given  to  each  team  in  a  TACOM  conference  room  on 
the  day  of  their  experiment.  The  researcher  presented  an  experiment  overview  of  the 
following:  general  experiment  purpose  and  objectives,  a  description  of  the  TFTS,  system 
requirements,  a  schedule  of  events,  and  a  description  of  test  conduct  and  protocol.  A 
copy  of  this  presentation  is  contained  in  Appendix  E. 

Test  Protocol 

Participants  were  asked  to  strictly  adhere  to  the  test  protocol  to  facilitate  data 
collection.  A  summarized  list  of  the  protocol  is  contained  in  the  test  plan  summary 
located  in  Appendix  C.  The  test  was  conducted  in  the  Design  Laboratory  at  U.  S.  Army 
TACOM.  Each  day,  a  experiment  design  team  moved  sequentially  between  technology 
stations  and  were  given  a  one-hour  time  limit  to  conduct  their  observations  and  design 
reviews.  Design  teams  were  asked  to  focus  on  one  TFTS  subassembly  at  a  time.  Order 
was  defined  following  the  design  sequence  of  the  Graeco-Latin  Square.  In  addition, 
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groups  were  told  that  there  were  at  least  three  problems  in  each  TFTS  subassembly. 
Participants  were  then  asked  to  identify  a  design  problem  sequentially,  and  to  resolve  the 
problem  before  moving  on  to  the  next  problem.  The  purpose  of  the  design  review  task 
was  to  reach  group  consensus  on  problem  identification  and  possible  resolution.  If  a 
design  team  could  not  solve  a  problem  it  was  required  that  team  members  must  agree  that 
the  resolution  is  not  solvable  at  this  time.  The  teams  were  then  allowed  to  move  to  the 
next  problem.  The  design  representative  in  each  experiment  team  was  designated  as  the 
team  leader  and  discussion  facilitator.  It  was  suggested  that  the  teams  use  the  technology 
in  anyway  they  felt  was  appropriate  for  the  problem  identification  and  resolution. 

For  consistency,  an  outside  single  independent  test  observer  served  as  a  data 
collector  for  each  day  and  moved  between  technology  stations  with  the  experiment 
design  teams.  This  observer,  a  senior-level  project  engineer,  was  given  brief  instructions 
on  subject  testing  methodologies  and  data  collection  procedures. 

The  study  researcher  served  three  roles  during  the  experimentation:  technology 
administrator,  technology  trainer,  and  secondary  observer.  As  the  technology 
administrator,  this  researcher  was  responsible  for  setting  up  each  experiment 
configuration  and  starting  up  each  system  prior  to  each  technology  experiment.  As  the 
technology  trainer,  the  researcher  taught  the  test  participants  basic  operating  procedures 
for  each  technology.  As  the  secondary  observer,  the  researcher  moved  between 
technology  stations  with  the  experiment  design  teams  and  conducted  outside  evaluations 
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of  each  experimental  session.  Appendix  F  contains  a  summary  of  the  comments  made 
during  these  observation. 

Technology  Familiarization 

The  researcher  conducted  a  standard  15-minute  system  familiarization  using  an 
alternate  model  at  each  technology  station  prior  to  testing.  All  experimental  team 
participants  received  technology  familiarization  that  provided  them  with  the  basic  skills 
necessary  to  operate  each  of  the  technologies. 

Empirical  Evaluation  of  the  Task 

The  independent  outside  test  observer  documented  the  performance  of  design 
teams  by  taking  continuous  time  measurements  of  the  start  and  completion  times  of  each 
problem  identified  and  resolved  by  the  team.  In  addition,  the  test  observer  documented 
how  the  design  teams  used  the  technologies  and  made  comments  on  anything  unusual 
that  occurred  that  may  have  impacted  the  data  during  the  experimentation.  The  data  was 
documented  on  a  quantitative  data  collection  sheet  designed  by  the  researcher  and  is 
included  as  Appendix  G. 

VE  Assessment  Survey  Two:  Individual  Technology 

A  VE  Assessment  Survey  Two  questionnaire  was  designed  by  the  researcher  to 
measure  the  strengths  and  weaknesses  of  the  visual  display  technologies  to  support  the 
design  review  process.  Refer  to  Appendix  H  for  details.  This  questionnaire  is  based  on  a 
fundamental  usability  assessment  method  and  used  Likert-like  questions.  Stuart  (1996) 
defines  this  type  of  assessment  as  rating  scales  with  which  users  retrospectively  express 
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their  subjective  satisfaction  with  specified  aspects  of  the  human-computer  interface. 
Participants  were  asked  to  make  their  assessment  on  a  I  to  5  scale,  where  the  value  of  one 
was  related  to  a  negative  feeling  and  a  value  of  5  was  related  to  a  positive  feeling.  The 
questionnaire  contained  29  questions  divided  into  four  sections:  training,  CDR  process, 
quality,  and  physiological  effects. 

Training  questions  were  designed  to  assess  whether  users  felt  they  had  an 
adequate  level  of  technology  familiarization  to  conduct  the  design  review  task. 

Questions  in  the  quality  section  were  based  a  Questionnaire  for  User  Interface 
Satisfaction  (QUIS)  created  at  the  Human-Computer  Interaction  Laboratory  at  the 
University  of  Maryland  (Chin,  Diehl,  &  Norman,  1988).  The  physiological  section  of  the 
questionnaire  focuses  on  the  mental  and  physical  state  of  the  participant’s  experience  in 
the  VE  after  exposure  to  the  four  treatment  technologies.  A  scoring  procedure  developed 
by  Kennedy,  Lane,  Lilienthal,  Berbaum,  and  Hettingger  (1992)  was  the  source  of  the  16 
symptoms  identified  in  the  questionnaire.  One  question  regarding  measures  of  perceived 
mental  workload  was  included  to  assess  how  mentally  demanding  the  technology  was  to 
the  test  participants.  This  question  was  based  on  a  modified  Cooper-Harper  (1969)  scale. 

To  gain  insight  into  the  value  of  the  technologies  and  possible  design 
improvements,  additional  user  feedback  questions  were  developed.  Participants  were 
asked  how  many  hours  a  day  they  would  be  willing  to  use  the  technology,  if  they  felt  the 
technology  was  more  beneficial  for  individual  use,  and  if  there  was  any  aspect  of  the 
visualization  tool  that  could  be  improved  to  enhance  CDR  tasks. 
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A  second  usability  assessment  approach  was  incorporated  in  VE  Assessment 
Survey  Two.  This  assessment  used  open-ended  questions,  which  allowed  users  to  record 
observations  and  subjective  comments  that  could  not  be  captured  with  a  Likert-like 
questionnaire  (Stuart,  1996;  Nielsen,  1993).  Space  was  provided  at  the  end  of  each 
section  in  the  questionnaire  for  participants  to  make  any  comments  regarding;  the  training 
they  received  prior  to  the  test  phase,  the  effect  of  the  visualization  tool  to  perform  CDR 
tasks,  the  quality  of  the  technology  as  it  applies  to  product  design,  and  their  physical  and 
emotional  state  after  using  each  technology.  The  questionnaire  was  administered  to  each 
test  participant  in  the  TACOM  Design  Lab  immediately  following  completion  of  each 
treatment  condition.  There  was  no  maximum  time  limit  for  VE  Assessment  Survey  Two 
completion. 

At  the  completion  of  each  technology  experiment  and  survey,  the  design  team 
moved  to  the  next  experimental  technology  according  to  their  assigned  test  sequence. 

This  staging  was  repeated  four  times,  until  each  design  team  completed  experimentation 
with  all  four-visual  display  technologies. 

VE  Assessment  Survey  Three:  Technology  Comparison 

At  the  end  of  the  entire  experiment,  and  after  each  team  had  completed 
experimentation  using  all  four  visual  display  technologies,  individual  participants  were 
asked  to  complete  VE  Assessment  Survey  Three  which  consisted  of  five  questions. 
Participants  were  individually  asked  to  compare  the  technologies  by  ranking  them  from 
one  to  four.  Where  a  value  of  one  was  considered  the  most  useful,  most  difficult,  most 
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practical,  most  helpful  in  stimulating  team  interactivity,  or  most  beneficial  for  developing 
team  consensus.  The  value  of  four  was  equated  to  the  least  performance  for  each 
technology. 

Space  was  also  provided  at  the  end  of  this  questionnaire  for  participants,  in 
retrospect  after  experiencing  all  four  visualizadon  technologies,  to  make  any  changes  to 
their  previous  responses  on  VE  Assessment  Survey  Two.  Appendix  I  contains  VE 
Assessment  Survey  Three  —  the  technology  comparison.  This  survey  was  administered  in 
the  U.S.  Army  TACOM  Design  Lab  immediately  following  completion  of  all  four- 
treatment  conditions.  There  was  no  maximum  time  limit  for  the  survey. 

Upon  completion  participants  were  asked  to  refrain  from  discussing  any  of  the 
visualization  technologies  with  any  other  colleagues  to  eliminate  the  possibility  of 
biasing  any  future  testing  or  experimentauon. 

Instrumentation 

Data  Measurement 

An  independent  outside  test  observer  using  a  digital  stop  watch  took  continuous 
time  measurements  of  the  start  and  complerion  rimes  of  each  problem  identified  and 
resolved  by  the  team.  Time  measures  were  taken  and  recorded  to  the  second.  Using  a 
single  independent  test  observer  maintained  consistency  and  reliability  of  the  data 
collected. 
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VE  Assessment  Surveys  One,  Two,  and  Three 

The  instruments  VE  Assessment  Surveys  One,  Two,  and  Three  were  constructed 
on  the  basis  of  the  review  of  related  literature  and  recommendations  from  dissertation 
committee  members.  However,  because  of  the  nature  of  the  instrument,  no  attempt  was 
made  to  establish  reliability  or  validity. 

Data  Analysis  Methodology 

Several  methodologies  were  planned  for  quantitative  and  qualitative  data 
analyses.  The  quantitative  analyses  employed  four-way  analysis  of  variance.  Dependent 
measures  were  the  number  of  design  errors  detected,  the  average  time  to  detect  a  design 
error,  and  the  average  time  to  resolve  a  detected  design  problem.  Significant  mean 
differences  were  analyzed  via  separate  post  hoc  analysis  utilizing  Bonferenoni  pairwise 
comparison  methodologies.  Descriptive  findings  and  Kruskal-Wallis  analysis  on  the 
cumulated  data  were  reported  for  qualitative  data  collected  in  VE  Assessment  Survey 
Two  --  Individual  Technology.  Data  collected  in  VE  Assessment  Survey  Three  -- 
Technology  Comparisons,  were  analyzed  using  non-parametric  Freidman  analyses. 
Significant  mean  differences  were  analyzed  via  separate  post  hoc  analysis  utilizing  a 
follow-on  pairwise  comparison  test  to  the  Freidman  statistic  method. 

Summary 

The  methods  and  procedure  for  the  experiment  followed  standard  laboratory 
research  testing  practices.  The  subjects  were  12  personnel  from  the  U.  S.  Army  TACOM 
who  served  as  subjects  to  test  the  hypotheses  of  the  effect  of  VE  visual  display 
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technologies  on  the  CDR  process.  This  experimental  design  focused  on  the  use  of  data  to 
show  the  differences  between  the  four  visual  display  technologies.  The  objective  of  the 
study  was  to  evaluate  the  use  and  value  of  four  different  commercial  visual  display 
systems  during  the  multi-functional  group  CDR  process  and  to  measure  how  this  process 
is  impacted  by  the  use  of  these  tools.  The  next  chapter  presents  analysis  of  the  data 
collected  from  the  experimentation. 
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CHAPTER  4 


DATA  ANALYSIS 

This  chapter  reports  the  results  of  the  analyses  performed  on  collected 
experimental  data.  This  study  employed  several  human-computer  interface  evaluation 
methods  to  assess  the  impact  of  the  four  tested  visual  display  technologies  on  the  CDR 
process.  This  chapter  is  organized  into  sections  based  on  the  sequence  of  data  collection 
methodologies  used.  These  are:  empirical  task  evaluation,  perception  (VE  Assessment 
Survey  Two),  and  preference  (VE  Assessment  Survey  Three).  The  latter  two  methods, 
described  in  Chapter  3,  used  questionnaires  designed  to  gather  supplemental  subjective 
data. 

Data  analyses  are  based  on  the  performance  of  12  test  participants  following  a 
4x4  Graeco-Latin  square  experimental  design.  The  four  independent  factors  were:  visual 
display  technologies  (A,  B,  C,  D),  design  teams  (Tt.  T2.T3.  T4),  Tracked  Fuel  Trailer 
System  (TFTS)  subassemblies  (1,  2,  3, 4),  and  experiment  order  (Ej.  Ei.  E3.  Ri). 

Empirical  evaluation  of  the  task.  The  three  dependent  measures  evaluated  were:  ( 1 )  the 
number  of  design  errors  detected,  (2)  the  average  time  to  detect  a  design  error,  and  (3)  the 
average  time  to  resolve  a  detected  design  error. 

Perception  (VE  Assessment  Survey  Two).  The  five  measures  for  test  participant 
perceptions  included:  (1)  technology  training,  (2)  the  CDR  process,  (3)  quality  of  the 
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CDR  process,  (4)  physiological  state  of  the  design  team  members  after  using  the  3-D 
visual  display  devices,  and  (5)  some  additional  questions  including  how  many  hours  each 
user  would  be  willing  to  use  the  technologies  and  if  they  thought  the  technologies  were 
more  beneficial  for  individual  use. 

Preferences  (VE  Assessment  Survey  Three).  The  five  measures  of  test  participant 
preferences  include:  (1)  usefulness,  (2)  difficulty,  (3)  practicality,  (4)  team  interaction, 
and  (5)  development  of  team  consensus  when  comparing  the  four  visual  display 
technologies. 

Results  from  the  empirical  evaluations  of  the  task  were  derived  from  four-way 
ANOVA  and  Bonferroni  pairwise  comparison  testing.  These  results  are  summarized  in 
Table  4.  Empirical  data  collected  during  the  experiment  are  contained  in  Appendix  J. 

The  p  values  shown  are  the  smallest  level  of  a  where  the  null  hypothesis  can  still  be 
rejected.  The  pairwise  comparison  portion  of  Table  4,  shown  as  horizontal  bars,  are  a 
pictorial  representation  of  the  similarities  between  the  four  visual  display  technologies. 
The  technologies  are  ranked  from  left  to  right  corresponding  to  best  to  worst 
performances. 

1 .  Error  Detection.  Design  teams  detected  more  errors  when  using  either  the 
stereoscopic  glasses  or  the  monoscopic  CRT  monitor  systems  and  the  least  when 
using  the  HMD  and  BOOM  system. 

2.  Time  to  Detect.  Design  teams  detected  design  problems  fastest  using  the  HMD  and 
slowest  when  using  the  stereoscopic  glasses  and  BOOM  systems. 
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Table  4. 


Four-Way  ANOVA  Data  Summary 


Dependent 

Variable 

Hypothesis 

p  value 

Pairwise  Comparison 

Best  ►  Worst 

C  D  A  B 

The 

number  of 

deign 

errors 

delected 

Technology: 

A  =  B  =  C  =  D 

.019 

Team: 

Ti  =  T2=T3  =  T4 

.046 

Subassembly: 

1 =2  =  3  =4 

.733 

Experiment  Order: 

E|  =  Ei  =  E3  =  E4 

.960 

A  D  C  B 

The 

average 

time  to 

deleci  a 

design 

error 

Technology: 

A  =  B  =  C  =  D 

.008 

Team: 

T,  =  T2=T3  =  T4 

.003 

Subassembly: 

1 =2  =  3  =4 

.008 

Experiment  Order: 

Ei  =  E2=  E3  =  E4 

.003 

A  B  C  D 

The 

average 

time  io 

resolve  a 

detected 

design 

problem 

Technology: 

A  =  B  =  C  =  D 

.777 

Not  Statistically 
Significant 

Team: 

t,  =  t2=t3  =  t4 

.340 

Subassembly: 

1  =2  =  3  =  4 

.551 

Experiment  Order: 

Ei  =  E2  =  E3  =  E4 

.748 

Note:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
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3.  Time  to  Resolve.  No  statistically  significant  differences  among  or  between  the  four 
treatment  visual  display  technologies  were  detected  for  the  dependent  variable  — 
average  resolution  time. 

Results  from  VE  Assessment  Survey  Two,  participants’  perception  of  the 
technologies  were  derived  from  nonparametric  Kruskal-Wallis  analyses,  follow-on 
pairwise  comparison  tests,  and  percentage  score  reporting.  The  results  from  Kruskal- 
Wallis  testing  are  summarized  in  Table  5.  The  p  values  shown  are  the  smallest  level  of  a 
where  the  null  hypothesis  can  still  be  rejected.  The  pairwise  comparison  portion  of 
Table  5,  shown  as  horizontal  bars,  are  a  pictorial  representation  of  the  similarities 
between  the  four  visual  display  technologies.  The  technologies  are  ranked  from  left  to 
right  corresponding  to  the  best  to  the  worst  performances. 

4.  Perception:  VE  Assessment  Survey  Two  —  the  individual  technology. 

a.  Training.  Based  on  the  frequency  of  participant  responses  in  the  training  section 
of  the  questionnaire,  participants  received  an  adequate  level  of  training  prior  to  the 
experiment. 

b.  CDR  Process.  Participants  perceived  that  the  HMD  and  BOOM  added  the  most 
value  to  the  CDR  process  and  stereoscopic  glasses  with  monitor  system  the  least. 

c.  Quality.  Participants  believed  the  quality  of  the  CDR  process  was  best  when  using 
the  HMD  and  monoscopic  CRT  monitor  systems  and  worst  when  using  the 
stereoscopic  glasses  with  monitor  system. 
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Table  5. 


Kruskal-Wallis  Data  Summary 


Dependent  Variable 

Hypothesis 

P 

value 

Pairwise  Comparison 

Best  ^  Worst 

A  B  D  C 

Effect  on  the  design  process 

G 

n 

U 

il 

CQ 

II 

< 

.000 

— 

A  D  B  C 

Effect  on  the  quality  of  the 
design  process 

A  =B  =  C  =  D 

.000 

— 

D  A  B  C 

Number  of  hours  willing  10 
use  the  technologies 

A  =B  =  C  =  D 

.029 

- - 

Are  the  technologies  more 
beneficial  for  individual 
use? 

A  =B  =  C  =  D 

.826 

Not  Statistically 
Significant 

Note:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 

d.  Phvsioloeical.  A  valid  statistical  assessment  of  the  mental  and  physical  state  of 
the  participants  could  not  be  made  because  sharing  hardware  resources  minimized  the 
exposure  time  for  individuals.  However,  eye  fatigue  was  experienced  when 
participants  used  the  stereoscopic  glasses. 

e.  Additional  Questions.  Test  participants  indicated  they  would  be  willing  to  use 
monoscopic  CRT  monitor  and  HMD  systems  for  longer  periods  of  time.  Their 
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responses  reveal  that  they  would  be  willing  to  use  a  traditional  monoscopic  monitor 
34%  longer  than  a  HMD,  and  55%  longer  than  either  the  BOOM  or  stereoscopic 
glasses.  A  second  question  asked  users  whether  they  thought  the  technologies  were 
better  suited  for  individual  rather  than  team  use.  Kruskal-Wallis  analysis  revealed  that 
there  was  not  sufficient  evidence  to  detect  differences  between  the  four  visual  display 
systems  for  this  study. 

Results  from  VE  Assessment  Survey  Three,  participant  preferences  of  the 
technologies,  were  derived  from  nonparametric  Friedman  tests  and  follow-on  pairwise 
comparison  testing  described  by  Daniel  (1978).  These  results  are  summarized  in  Table  6. 
The  p  values  shown  are  the  smallest  level  of  a  where  the  null  hypothesis  can  still  be 
rejected.  The  pairwise  comparison  portion  of  Table  6,  shown  as  horizontal  bars,  are  a 
pictorial  representation  of  the  similarities  between  the  visual  display  technologies.  The 
technologies  are  ranked  from  left  to  right  corresponding  to  best  to  worst  performances. 

5.  Preferences:  VE  Assessment  Survey  Three  —  technology  comparisons. 

a.  Usefulness.  Test  participants  ranked  the  HMD  and  monoscopic  CRT  as  the  most 
useful  technologies  and  BOOM  and  stereoscopic  glasses  as  the  least. 

b.  Difficulty.  Results  for  the  difficulty  in  using  the  technology  dependent  variable 
revealed  that  no  statistically  significant  differences  exist  between  the  four  visual 
display  technologies. 
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c.  Practicality.  Test  participants  indicate  that  the  monoscopic  CRT  monitor  system 
was  the  most  practical  technology  and  that  it  differed  significantly  from  the  other 
visual  display  systems. 


Table  6. 

Freidman  Analysis  Data  Summary 


Dependent  Variable 

Hypothesis 

P 

value 

Pairwise  Comparison 

Best  ^  ^Worst 

A  D  B  C 

Usefulness  on  the  CDR 
process 

A  =  B  =  C  =  D 

.042 

-  - 

Difficulty  using  the 
technology  for  CDR 

A  =  B  =  C  =  D 

.220 

Not  Statistically 
Significant 

D  A  C  B 

Practicality  of  using  the 
technology  for  CDR 

A  =B  =  C  =  D 

.001 

— 

D  A  C  B 

Stimulation  of  team 
interactivity  using  the 
technology  for  CDR 

A=B=C=D 

.021 

— 

D  A  C  B 

Usefulness  for  developing 
team  consensus  using  the 
technology  for  CDR 

> 

II 

w 

II 

Cj 

II 

D 

.006 

Note:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
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d.  Team  Interactivity.  When  ranking  usefulness  for  team  interactivity,  participants 
felt  that  the  monoscopic  CRT  monitor  and  HMD  systems  were  the  most  useful  for 
team  interaction  and  differed  significantly  from  both  the  BOOM  and  stereoscopic 
glasses  systems. 

e.  Team  Consensus.  Test  participants  thought  that  the  monoscopic  CRT  monitor 
system  was  the  most  beneficial  for  helping  design  teams  formulate  consensus. 

Results  also  reveal  that  the  HMD  differed  significantly  and  performed  better  than  the 
BOOM  and  stereoscopic  glasses  systems. 

The  next  sections  provide  more  detailed  evaluations  and  discussions  of  the  data 
collected  in  this  study. 

Number  of  Design  Errors  Detected 

Hypothesis  #1:  There  will  be  no  significant  difference  among  the  four  visual 
display  technologies  (A,  B,  C,  D),  among  the  four  design  teams  (T j.  Ti.  T3.  T4),  among  the 
four  lh IS  subassemblies  (l,  2,  3, 4),  and  among  the  four  experiment  orders  (E|.  E;.  E3. 
E4)  in  the  number  of  design  errors  detected  during  a  CDR  task. 

The  first  hypothesis  was  tested  using  a  four-way  ANOVA.  The  criterion  variable 
used  to  assess  the  number  of  design  errors  detected  was  the  numerical  count  of  design 
errors  identified  by  the  design  teams  when  conducting  a  CDR  of  the  TFTS.  Results  of  the 
four- way  ANOVA  concluded  rejection  of  the  null  hypothesis  indicating  significant 
differences  (p<.  05)  exist  between  the  four  visual  display  technologies  and  the  four 
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design  teams.  The  results  also  showed  that  there  is  not  sufficient  evidence  that 
differences  exist  between  the  four  subassemblies  and  experimental  order. 

A  post  hoc  Bonferroni  pairwise  comparison  test  was  used  to  determine  the 
differences  found  between  four  visual  display  technologies.  This  analysis  revealed  that 
design  teams  detected  more  design  errors  when  using  either  the  stereoscopic  glasses  or 
monoscopic  CRT  monitor  systems  and  the  least  when  using  HMD  or  the  BOOM 
technologies.  Appendix  K  contains  means  and  standard  deviations  of  the  number  of 
design  errors  detected  for  each  treatment. 

The  average  number  of  design  errors  detected  for  each  experiment  variable  are 
listed  in  Table  7.  On  average,  design  teams  were  able  to  detect  more  design  errors  when 
using  either  stereoscopic  glasses  or  monoscopic  CRT  monitor  systems.  This  table  also 
shows  that  on  average  design  team  one  (TO  detected  the  least  number  of  errors.  This 
finding  could  be  explained  because  this  team  had  the  longest  error  detection  time  and  did 
not  fully  identify  all  the  design  errors  before  completing  their  evaluation.  Table  7  also 
shows  the  average  number  of  errors  detected  for  each  TFTS  subassemblies  and  each 
experiment  order.  These  support  ANOVA  findings  that  no  significant  differences  exist 
between  the  TFTS  subassemblies  and  experiment  order. 

Graphs  of  the  least  square  means  for  experiment  variables  vs.  the  dependent 
measure  (number  of  design  errors  detected)  are  shown  in  Figure  15.  The  greater  number 
of  errors  detected  is  defined  as  the  best  team  performance.  The  graphs  pictorially  support 
findings  of  the  four-way  ANOVA  and  Bonferroni  tests,  indicating  that  design  teams 
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detect  the  greatest  amount  of  design  errors  when  using  stereoscopic  glasses  and  the  least 
amount  when  using  the  BOOM. 


Table  7. 

Average  Number  of  Design  Errors  Detected  for  Each  Experiment  Variable 


Variable 

Average  Number  of 
Design  Errors 

Visual  Display  Technology 

A 

5.25 

B 

5.00 

C 

9.25 

D 

8.00 

Design  Team 

T, 

4.75 

t2 

7.50 

t3 

7.00 

t4 

8.25 

TFTS  Subassembly 

1 

6.75 

2 

6.75 

3 

7.00 

4 

7.00 

Experiment  Order 

E, 

6.75 

e2 

7.25 

e3 

7.00 

E4 

6.50 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Larger  number  of  errors  =  Better  Performance. 
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Figure  15.  Graphs  Of  the  Least  Square  Means  for  the  Number  of  Design  Errors  Detected. 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Large  number  of  errors  detected  =  best  performance. 


Time  to  Detect 


Hypothesis  #2:  There  will  be  no  significant  difference  among  the  four  visual 
display  technologies  (A,  B,  C,  D),  among  the  four  design  teams  (Tj.T2.T3.  T4),  among  the 
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four  TFTS  subassemblies  (1,  2, 3, 4),  and  among  the  four  expenment  orders  (Ei.  Et.  E3. 
E»)  in  the  average  time  it  takes  to  detect  a  design  error  during  a  CDR  task. 

The  second  hypothesis  was  tested  using  a  four-way  ANOVA.  The  criterion 
variable  used  to  assess  the  time  to  detect  a  design  error  was  the  average  number  of 
seconds  each  group  required  to  detect  TFTS  subassembly  design  errors  when  conducting 
the  CDR.  An  independent  data  collector  made  continuous  time  measurements  of  both  the 
start  and  completion  times  of  each  design  error  identified  and  resolved  by  the  teams.  The 
time  to  detect  a  design  error  was  calculated  as  the  difference  between  the  time  of 
resolving  a  problem  and  the  time  of  detecting  a  new  design  problem.  To  facilitate  data 
collection,  design  teams  were  asked  to  follow  a  testing  protocol  that  required  them  to 
sequentially  identify  design  problems,  flaws,  and  issues.  They  were  asked  not  to  move 
on  to  another  design  problem  until  they  developed  a  team  consensus  on  the  current 
problem. 

The  results  of  the  four-way  ANOVA  indicate  a  significant  difference  (p<  .01 ) 
between  the  four  visual  display  technologies,  four  design  teams,  four  TFTS 
subassemblies,  and  four  experiment  orders  in  the  average  time  to  detect  a  design  error. 
Appendix  L  contains  means,  and  standard  deviations  of  the  average  time  to  detect  a 
design  error  for  each  treatment. 

A  post  hoc  Bonferroni  pairwise  comparison  test  was  used  to  determine  the 
differences  found  between  the  four  visual  display  technologies.  This  analysis  revealed 
that  the  average  time  to  detect  a  design  problem  using  the  HMD  were  significantly 
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different  and  better  than  average  detection  times  when  design  teams  used  any  of  the  other 
visual  display  technologies.  The  post  hoc  analysis  also  revealed  that  the  monoscopic  CRT 
monitor  differed  significantly  and  better  than  the  BOOM  and  stereoscopic  glasses 
configurations  which,  were  found  to  be  similar  and  performed  the  worst. 

Table  8  contains  the  mean  detection  times  for  each  experiment  variable.  By 
definition,  the  smallest  detection  time  is  associated  with  best  performance  On  average 
design  teams  were  able  to  detect  design  errors  faster  using  either  the  HMD  or 
monoscopic  CRT  monitor  systems. 

Graphs  of  the  least  square  means  for  experiment  independent  variables  vs.  the 
dependent  measure  (average  time  to  detect  a  design  error)  are  shown  in  Figure  16.  These 
graphs  pictorially  show  the  differences  between  the  four  visual  display  systems.  This 
supports  conclusions  drawn  from  post  hoc  testing.  The  graph  indicates  that  design  teams 
detect  design  errors  fastest  when  using  the  HMD  and  the  slowest  when  using  the  BOOM 
and  stereoscopic  glasses  systems. 

Note  that  the  results  for  the  number  of  errors  detected  appears  to  be  inconsistent 
with  the  results  obtained  for  the  average  detection  time.  The  stereoscopic  glasses  yielded 
the  best  performance  when  measuring  the  design  errors  detected  and  in  contrast  yielded 
worse  performance  when  measuring  for  the  dependent  variable  average  detection  time. 
Similarly,  the  HMD  yielded  the  best  performance  for  average  detection  time  and  a  worse 
performance  for  the  design  errors  detected.  On  the  surface  the  results  of  the  data  appear 
to  be  inconsistent,  but  they  were  based  on  the  collected  data.  These  findings  indicate  that 
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in  a  team  setting,  the  technology  that  promotes  natural  dialogue  yields  the  most  number 
of  errors  detected,  but  the  technology  that  provides  the  users  with  the  best  sense  of 
presence  in  the  VE  yields  the  fastest  error  detection  times. 

Table  8. 

Mean  Detection  Time  (Seconds)  for  Each  Variable 


Variable 

Average  Detection 
Time  (sec) 

Visual  Display  Technology 

A 

51.945 

B 

66.477 

C 

64.468 

D 

57.777 

Design  Team 

Ti 

69.453 

T-> 

63.788 

t3 

48.750 

T4 

58.678 

TFTS  Subassembly 

1 

51.915 

2 

59.402 

3 

61.615 

4 

67.735 

Experiment  Order 

Hi 

51.458 

Ei 

57.053 

e3 

73.930 

E4 

58.228 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Lower  detection  time  =  greater  performance. 
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Figure  1 6.  Graphs  of  the  Least  Square  Means  for  the  Average  Time  to  Detect  a  Design 

Error 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Lower  detection  time  =  better  performance. 

Time  to  Resolve 

Hypothesis  #3:  There  will  be  no  significant  difference  among  the  four  visual  display 
technologies  (A,  B,  C,  D),  among  the  four  design  teams  (T1.T2.T3.  T4),  among  the  four 


138 


subassemblies  (1,  2,  3, 4),  and  among  the  four  experiment  orders  (Et.  E2.  E3.  E4)  in  the 
average  time  to  resolve  an  identified  design  error. 

The  third  hypothesis  was  tested  using  a  four-way  ANOVA.  The  criterion  variable 
used  to  assess  the  time  to  resolve  a  design  error  was  the  average  number  of  seconds  each 
group  required  to  resolve  an  identified  design  error  or  flaw,  achieving  group  consensus 
when  conducting  a  CDR  of  the  TFTS  model.  An  independent  data  collector  made 
continuous  time  measurements  of  both  the  start  and  completion  time  of  each  design  error 
identified  and  resolved  by  the  groups.  The  design  error  resolution  time  was  calculated  as 
the  difference  between  the  time  of  identifying  a  design  problem  and  the  time  of  reaching 
a  design  team  consensus  on  the  problem’s  resolution.  The  testing  protocol  required  that 
design  teams  sequentially  identify  design  problems.  To  facilitate  data  collection  design 
teams  were  asked  not  to  move  on  to  a  new  design  problem  until  they  developed  a 
consensus  on  the  current  problem  identified. 

The  results  of  the  four-way  ANOVA  indicated  no  significant  differences  (p>.7) 
between  the  four  visual  display  technologies,  the  four  design  teams,  the  four  TFTS 
subassemblies,  and  the  four  experiment  orders  for  the  average  time  to  resolve  a  design 
problem.  Appendix  M  contains  means,  and  standard  deviations  of  the  average  detection 
time  for  each  experiment  treatment. 

This  study  failed  to  find  differences  between  the  four  visual  display  technologies 
for  the  problem  resolution  time  dependent  variable.  Investigation  into  probable  causes 
revealed  widely  disparate  standard  deviation  values.  The  ANOVA  F  test  used  to  test  the 
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null  hypothesis  of  equality  of  means  was  based  on  the  assumption  that  the  populations  are 
normally  distributed  with  a  common  variance  (Mendenhall  &  Sincich,  1988).  The 
disparate  variances  could  have  caused  the  insignificant  results  obtained  from  the  four¬ 
way  ANOVA,  therefore,  a  nonparametric  Kruskal-Wallis  test  was  conducted  to  eliminate 
the  potential  variance  problem.  This  test  is  similar  to  the  ANOVA  F  test  except  that  the 
population  is  not  assumed  to  have  a  normal  distribution  with  a  common  variance. 
Mendenhall  &  Sincich  (1988)  state  that  the  primary  advantage  of  this  test  is  that  no 
assumptions  are  made  about  the  nature  of  the  sampled  population. 

The  Kruskal-Wallis  test  was  conducted  to  test  the  null  hypothesis  that  the  four 
\  iS'“\l  display  technologies  possess  the  same  probability  distribution  against  the 
alternative  hypothesis  that  the  distributions  differ  in  location.  The  results  of  the  Kruskal- 
Wallis  test  revealed  that  there  is  insufficient  evidence  to  indicate  a  difference  in  location 
among  the  distributions  of  design  error  resolution  ume  for  the  four  visual  display 
technologies  (p>.  95).  No  significant  differences  between  the  four  visual  display 
technologies  were  found,  indicating  that  this  study  failed  to  detect  any  differences  for  the 
dependent  variable  --  problem  resolution  time.  Table  9  contains  the  mean  resolution 
times  for  each  independent  variable. 
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Table  9. 


Mean  Resolution  Time  (Seconds)  for  Each  Variable 


Variable 

Average  Resolution 
Time  (seconds) 

Visual  Display  Technology 

A 

59.403 

B 

85.503 

C 

77.320 

D 

59.595 

Design  Team 

. 

T, 

1 10.925 

T-> 

49.268 

t3 

63.320 

t4 

58.307 

TFTS  Subassembly 

1 

57.235 

2\ 

56.128 

3 

98.368 

~ 

70.090 

Experiment  Order 

Hi 

85.648 

e2 

69.125 

e3 

74.750 

E4 

52.298 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Lower  average  resolution  time  =  greater  performance. 


Perception:  VE  Assessment  Survey  Two  —  The  Individual  Technology 

A  29-item  questionnaire,  VE  Assessment  Survey  Two:  Individual  Technology, 
was  provided  to  each  participant  at  the  end  of  each  experiment  treatment  and  is  included 
as  Appendix  H.  The  questionnaire  was  designed  to  analyze  test  participants’  perceptions 
of  the  four  visual  display  technologies  on  the  CDR 
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process.  The  assessment  was  divided  into  four  sections:  training,  CDR  process,  quality, 
and  physiological.  Participants  were  asked  to  make  their  assessment  on  a  1  to  5  Likert 
scale,  because  of  the  small  sample  size  in  this  study  the  data  was  condensed  into  two 
values.  The  scale  values  of  1 ,  2,  and  3  were  combined  to  a  value  of  1  and  scale  values  4 
and  5  were  combined  to  a  value  of  5.  These  values  were  then  equated  to  No  and  Yes, 
respectively.  Sparse  data  restricted  the  ability  to  conduct  Chi-square  analyses.  Instead 
questions  in  each  section  of  the  questionnaire  were  combined  into  one  and  were 
evaluated  using  Kruskal-Wallis  analysis.  Percentage  score  reporting  also  was  used  to 
support  the  analysis.  In  addition,  supplemental  data  were  collected  by  providing  space  at 
the  end  of  each  section  and  at  the  end  of  the  questionnaire  for  participants  to  make 
comments.  A  compilation  of  user  comments  is  provided  in  Appendix  N. 

Technology  Training 

Hypothesis  #4:  There  will  be  no  practical  difference  in  the  perception  of 
technology  training  and  familiarization  among  the  four  visual  display  technologies. 

The  purpose  of  the  technology  training  section  of  VE  Assessment  Survey  Two  — 
Individual  Technology  was  to  verify  technology  familiarization  and  to  assess  if  test 
participants  felt  they  were  adequately  prepared  to  conduct  the  CDR  task  using  the  four 
visual  display  technologies.  Table  10  includes  participant  response  frequency  by 
technology  for  each  question  in  the  training  section  of  the  survey. 

Few  negative  responses  were  obtained  in  this  section  of  the  assessment,  therefore, 
valid  statistical  methods  could  not  be  conducted.  For  that  reason,  the  data  was  analyzed 
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using  percentage  scores.  Based  on  higher  percentage  of  yes  responses  in  the  training 
section  of  the  questionnaire,  it  was  concluded  that  test  participants  received  an  adequate 
level  of  training  and  had  the  competency  to  use  and  operate  the  technologies  prior  to  the 
experimentation.  For  example,  87.5%  (42/48)  “found  the  training  to  be  sufficient’XQl), 
85.4%  (41/48)  “thought  that  they  fully  understood  the  correct  operating  procedures  for 
the  technologies”  (Q2),  and  only  20.8%  (10/48)  “thought  they  were  not  prepared"  (Q3). 
Ninety-one  percent  (1 1/12)  of  the  test  participants  felt  that  BOOM  was  easy  to  operate 
compared  to  83.3%  (10/12)  for  the  HMD.  Only  50%  (6/12)  of  the  participants  responded 
that  the  stereoscopic  glasses  configuration  were  easy  to  operate  (Q4). 

Table  10. 


Frequency  of  Training  Responses 


VISUAL  DISPLAY  TECHNOLOGY 

Effects  of  Training 

A 

B 

c 

D 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Qi 

Was  the  training  sufficient? 

11 

1 

11 

1 

9 

3 

11 

1 

Q2 

Did  you  fully  understand  the  correct 
operating  procedures? 

12 

0 

11 

1 

9 

3 

9 

3 

Q3 

Were  you  prepared? 

a 

1 

11 

1 

8 

4 

8 

4 

04 

Learning  to  operate  the  system  was  easy? 

10 

2 

11 

1 

6 

6 

8 

4 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Y=  Yes,  N=No. 

As  can  be  seen  from  Table  10,  more  negative  responses  were  associated  with  the 


stereoscopic  glasses  and  monoscopic  CRT  monitor  configurations.  This  could  be 
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attributed  to  the  navigation  method  employed  in  these  two  configurations  --  a  mouse. 
Movement  in  the  environment  when  using  a  mouse  may  have  required  additional 
technology  familiarization. 

Concept  Design  Review  Process 

Hypothesis  #5:  There  will  be  no  practical  difference  in  the  perception  of  the  value 
of  the  four  visual  display  technologies  on  the  CDR  process. 

The  CDR  process  section  of  VE  Assessment  Survey  Two  was  designed  to  assess 
the  impact  the  four  visual  display  technologies  had  on  the  CDR  process.  Table  1 1 
includes  the  frequency  per  technology  for  each  item  included  on  the  process  section  of 
the  questionnaire. 

Cumulative  participant  responses  for  the  questions  in  the  design  process  section 
were  used  to  obtain  an  overall  design  process  comparison.  A  Kruskal-Wallis  analysis  was 
then  conducted  on  the  independent  variable,  technology.  Results  of  the  Kruskal-Wallis 
test  reveal  significant  differences  (pc.  01)  between  the  four  visual  display  technologies 
for  the  dependent  variable  —  effect  on  the  design  process.  A  follow-on  pairwise 
comparison  procedure  as  described  by  Conover  (1980)  was  conducted  to  determine  the 
differences  found  in  the  Kruskal-Wallis  analysis.  Pairwise  comparison  results  indicate 
that  the  HMD,  BOOM,  and  monoscopic  CRT  monitor  differed  significantly  (a=0.05) 
from  the  stereoscopic  glasses.  Participants  perceived  that  the  HMD  and  BOOM  added  the 
most  value  to  the  CDR  process  and  stereoscopic  glasses  with  monitor  system  the  least. 
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Table  11. 


Frequency  of  Concept  Design  Review  Process  Responses 


VISUAL  DISPLAY  TECHNOLOGY 

Effects  on  the  Design  Review 
Process 

A 

B 

c 

D 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Q5 

Did  the  technology  assist  you  in 
conducting  a  team  concept  review? 

ll 

1 

10 

2 

2 

10 

10 

2 

Q6 

Did  the  technology  assist  in  detecting 
design  problems  or  issues? 

10 

2 

10 

2 

5 

7 

7 

5 

Q7 

Was  the  technology  helpful  in 
describing  issues  to  team  members? 

10 

2 

8 

3 

7 

5 

8 

4 

Q8 

Did  the  technology  stimulate  creativity 
&  problem  solving? 

10 

2 

10 

2 

6 

6 

7 

5 

Q9 

Did  the  technology  assist  team 
interaction  &  discussion? 

10 

2 

10 

2 

8 

4 

10 

2 

Q10 

Did  the  technology  help  develop  a  team 
consensus? 

9 

2 

9 

3 

5 

7 

8 

4 

QH 

Did  the  technology  assist  in  providing  a 
better  understanding  of  the  concept 
(configuration)? 

10 

2 

10 

2 

3 

9 

9 

3 

Q12 

Did  the  technology  provide  easy 
recognition  of  form  &  pattern? 

10 

2 

10 

2 

5 

7 

8 

4 

Q13 

If  this  technology  was  a  standard  for 
concept  reviews  would  the  product 
development  time  decrease? 

9 

3 

7 

5 

4 

8 

8 

4 

Q14 

If  this  technology  was  a  standard  for 
concept  reviews  would  the  overall 
product  quality  improve? 

10 

2 

7 

5 

5 

7 

7 

5 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
"  Y=Yes,  N=No. 


A  bar  chart  showing  the  dependent  measure,  the  average  value  on  the  CDR 
process,  for  each  visual  display  technology  is  provided  in  Figure  17.  This  graph  supports 
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findings  found  in  the  Kruskal-Wallis  test,  i.e.,  users  felt  the  stereoscopic  glasses  had  a 
negative  effect  on  design  team  processes.  In  contrast,  the  use  of  the  other  three 
technologies  had  a  more  positive  effect  or  value  on  design  review  process. 


Figure  17.  Bar  Chart  of  Perceived  Value  on  Design  Process 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Scale  larger  value  =  better  performance. 

Five  of  the  questions  in  the  CDR  process  section  of  VE  Assessment  Survey  Two 
addressed  whether  the  technologies  assisted  in  team  activity  (Q5),  team  discussion  (Q7), 
team  problem  solving  (Q8),  team  interaction  (Q9),  and  team  consensus  (Q10). 
Participants’  subjective  responses  indicate  that  the  use  of  the  visual  display  technologies 
did  have  an  effect  on  the  CDR  process.  Overall,  the  participants  felt  that  stereoscopic 
glasses  were  the  least  helpful.  Only  16%  (2/12)  felt  that  they  “assisted  the  design 
process”  (Q5),  only  58%  (7/12)  thought  they  “helped  describe  problems  to  other  team 
members”  (Q7),  and  only  41%  (5/12)  felt  they  “assisted  in  developing  team  consensus” 
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(Q10).  In  contrast,  users  felt  that  the  HMD,  BOOM,  and  monoscopic  CRT  monitor  were 
more  helpful  and  useful  to  the  CDR  process.  Ninety-one  percent  ( 1 1/12)  of  the 
participants  believed  the  “the  HMD  assisted  in  the  process,  and  83.3%  (10/12)  thought 
that  the  HMD,  BOOM,  and  monoscopic  CRT  monitor  assisted  in  team  interaction  and 
discussion. 

Three  questions  focused  on  how  the  technologies  effected  the  detection  of  design 
problems  (Q6),  whether  the  technologies  provided  an  improved  understanding  of  the 
design  (Q 11),  and  if  the  technologies  helped  in  recognition  of  forms  and  patterns  (Q12). 
Test  participants’  subjective  responses  indicate  that  the  use  of  the  visual  display 
technologies  did  have  an  effect  on  the  design  process.  Eighty-three  percent  (10/12) 
responded  that  either  the  HMD  or  BOOM  helped  them  identify  and  detect  design 
problems  (Q6).  In  contrast,  only  41.67%  (5/12)  felt  that  stereoscopic  glasses  and  58.33% 
(7/12)  thought  using  the  monoscopic  CRT  monitor  aided  in  problem  detection. 

Users  were  asked  whether  they  felt  that  the  four  visual  display  technologies  would 
help  decrease  product  development  times  (Q13).  Seventy-five  percent  (9/12)  felt  that  the 
HMD  could  help  reduce  development  time  and  only  33.33%  (4/12)  felt  that  the 
stereoscopic  glasses  could  decrease  development  times.  Users  were  also  asked  whether 
they  felt  that  the  four  visual  display  technologies  would  help  improve  overall  product 
quality  (Q14).  Responses  indicate  that  83.33%  (10/12)  felt  that  the  quality  would  be 
improved  when  using  the  HMD,  58.33%  (7/12)  thought  that  the  BOOM  and  monoscopic 
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CRT  monitor  would  make  improvements  to  quality,  and  41 .67%  (5/12)  felt  that 
stereoscopic  glasses  could  impact  quality. 

Quality 

Hypothesis  #6:  There  will  be  no  practical  difference  in  the  perception  of  the  effect 
on  the  quality  of  the  design  process  among  the  four  visual  display  technologies. 

The  quality  section  of  VE  Assessment  Survey  Two  was  used  to  examine  how  the 
quality  of  the  CDR  process  was  effected  by  the  four  visual  display  systems.  Cumulative 
participant  responses  for  questions  in  the  quality  section  of  VE  Assessment  Survey  Two 
were  used  to  evaluate  the  effect  of  the  display  systems  on  the  factor  of  “quality.”  A 
Kruskal -Wallis  analysis  was  then  conducted  on  the  independent  variable  —  technology. 
Results  of  the  Kruskal-Wallis  test  revealed  significant  differences  (p<.  01)  between  the 
four  visual  display  technologies  for  the  dependent  variable  -  overall  quality.  A  post  hoc 
pairwise  comparison  test  (Conover,  1980)  was  conducted  to  determine  the  differences 
found  in  the  Kruskal-Wallis  test.  Results  indicate  that  the  overall  quality  of  the 
stereoscopic  glasses  differed  significantly  and  were  perceived  worst  than  the  other 
technologies.  Participants  believed  the  quality  of  the  CDR  process  was  best  when  using 
the  HMD  and  monoscopic  CRT  monitor  systems. 

Table  12  includes  the  frequency  per  technology  for  each  item  included  on  the 
quality  section  of  the  questionnaire.  In  some  cases,  test  participants  did  not  respond  to  the 
question,  therefore  not  all  questions  have  a  total  of  12  responses. 
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Table  12. 


Frequency  of  Quality  Responses 


1 

VISUAL  DISPLAY  TECHNOLOGY 

Effects  on  the  Quality  of  the 
Design  Review  Process 

A 

B 

c 

D 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Q15 

The  technology  is  helpful  as  a  platform 
for  team  discussion 

10 

2 

9 

3 

5 

7 

9 

3 

Q16 

The  technology  is  totally  useful  to  the 
product  design  review  process 

10 

2 

8 

4 

3 

9 

10 

2 

Q17 

Time  was  well  spent  using  the  technology 

10 

2 

9 

3 

5 

7 

8 

4 

QI8 

Adequate  time  was  given  to  use  the 
technology 

9 

3 

9 

3 

1 

5 

9 

3 

QI9 

No  time  was  wasted  using  the  technology 

10 

2 

7 

5 

6 

6 

8 

4 

Q20 

The  technology  had  a  positive  effect  on 
the  quality  of  the  design  review  process 

9 

3 

8 

4 

4 

8 

8 

4 

Q2I 

Overall,  the  technology  was  wonderful 

10 

2 

6 

6 

2 

10 

9 

3 

Q22 

Overall,  the  technology  was  satisfying 

8 

3 

6 

4 

2 

10 

9 

3 

Q23 

Overall,  the  technology  was  stimulating 

10 

1 

8 

3 

3 

9 

12 

0 

Q24 

Operating  the  system  was  easy  to  use 

10 

2 

7 

3 

3 

9 

7 

5 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Y=Yes,  N=No. 


A  bar  chart  graph  of  the  dependent  measure,  quality  of  the  design  review  process 
for  each  visual  display  technology  is  provided  in  Figure  18.  This  graph  supports  the 
findings  from  the  Kraskal-Wallis  test.  Users  felt  that  the  stereoscopic  glasses  had  the 
least  effect  or  value  on  quality  of  CDR  processes.  In  contrast,  they  felt  that  the  other 
three  technologies  had  a  more  positive  effect  on  overall  quality. 
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A  B  C  D 


Visual  Display  Technology 
Figure  18.  Bar  Chart  of  Perceived  Value  on  Quality 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 

Scale  higher  value  =  better  performance. 

Participants  felt  that  the  stereoscopic  glasses  were  the  least  helpful  for  group 
discussion  (Q15),  the  least  useful  to  the  design  review  process  (Q16),  and  had  little 
positive  effect  on  the  design  review  process  (Q20).  With  regard  to  Q20,  only  33.3% 
(4/12)  of  the  participants  felt  that  stereoscopic  glasses  impacted  the  quality  of  the  design 
process  in  a  positive  way,  while  81.8%  (9/12)  thought  the  HMD  and  66.67%  (8/12) 
thought  BOOM  and  monoscopic  CRT  monitor  positively  enhanced  the  quality  of  the 
process. 

Three  time-related  questions  addressed  how  well  the  time  was  spent  (Q17),  if 
enough  time  was  given  to  conduct  the  exercise  (Q18),  and  if  any  time  was  wasted  using 
the  technology  (Q19).  Eighty-three  percent  (10/12)  of  the  users  felt  that  the  time  using 
the  HMD  was  well  spent,  and  only  41.67%  (5/12)  thought  the  time  spent  using 
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stereoscopic  glasses  system  was  worthwhile.  Seventy-five  percent  (9/12)  of  the  design 
team  participants  felt  that  adequate  time  was  given  for  the  HMD,  BOOM,  and 
monoscopic  CRT  monitor  systems.  Users  felt  that  some  time  was  wasted  using  the 
stereoscopic  glasses  and  monoscopic  CRT  monitor  systems.  Again  these  descriptive 
statistical  findings  may  be  attributable  to  the  navigation  systems  used  in  these  two 
configurations. 

Four  of  the  questions  in  the  quality  section  of  VE  Assessment  Survey  Two  are 
related  to  participant  perceptions  of  the  technologies.  Participants  felt  that  stereoscopic 
glasses  were  the  least  wonderful  to  use  (Q21)  the  least  satisfying  to  use  (Q22),  the  least 
stimulating  to  use  (Q23),  and  the  most  difficult  to  operate  (Q24).  Only  16.67%  (2/12) 
thought  that  using  stereoscopic  glasses  were  wonderful  and  satisfying  and  only  25% 

(3/12)  thought  it  was  stimuladng  and  easy  to  operate.  In  contrast,  83.3%  (10/12) 
participants  felt  that  the  HMD  was  satisfying  and  90.91%  (10/1 1)  felt  it  was  stimulating. 

Physiological  Effects 

Hypothesis  #7:  There  is  no  practical  difference  in  the  physiological  effect 
experienced  by  the  test  participants  among  the  four  visual  display  technologies. 

The  physiological  section  of  VE  Assessment  Survey  Two  addressed  the  mental 
and  physical  state  of  the  design  team  participants  after  exposure  to  four  visual  display 
technologies.  A  valid  statistical  assessment  of  the  physiological  effects  could  not  be  made 
because  design  teams  had  too  short  of  an  exposure  time  when  using  the  HMD  and 
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BOOM  system.  This  was  due  to  the  need  to  share  the  hardware  resource  inherent  to  these 
technologies.  Instead  an  analysis  of  descriptive  statistics  was  conducted. 

One  question  regarding  measures  of  perceived  mental  workload  assessed  how 
mentally  demanding  the  technology  was  to  test  participants.  Table  13  includes  the 
frequency  per  technology  for  those  responses.  Overall,  82.6%  (38/46)  of  the  participants’ 
felt  that  the  technologies  were  not  mentally  demanding,  however,  they  did  feel  that  the 
most  mentally  demanding  of  the  technologies  was  the  monoscopic  CRT  monitor. 

Table  13. 


Frequency  of  Mental  Workload  Responses 


VISUAL  DISPLAY  TECHNOLOGY 

Physiological  Effect 

A 

B 

c 

D 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Q25 

Is  the  system  mentally  demanding? 

2 

8 

1 

11 

0 

12 

5 

7 

Note:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 


Design  participants  were  also  asked  to  identify  how  many  of  16  simulator 
sickness  symptoms  they  experienced  after  exposure  to  the  four  visual  display 
technologies.  These  symptoms  are  based  on  a  scoring  procedure  developed  by  Kennedy 
et  al.  (1992),  and  are  listed  the  VE  Assessment  Survey  Two  questionnaire  contained  in 
Appendix  H.  Table  14  provides  the  number  of  symptoms  experienced  and  the  number  of 
test  participants  who  reported  these  findings. 
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Table  14. 


Frequency  of  Physiological  Symptoms 


VISUAL  DISPLAY  TECHNOLOGY 

Physiological  Effect 

A 

B 

c 

D 

Q26 

The  number  of  physiological  symptoms 
experienced 

1 

4 

9 

0 

The  number  of  persons  experiencing 
physiological  symptoms 

1 

2 

4 

0 

Note:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 


The  majority  of  the  participants  did  not  experience  any  significant  physiological 
symptoms  associated  with  exposure  to  the  test  technologies.  However,  33.3%  (4/12)  of 
the  participants  experienced  some  symptoms  such  as  eyestrain,  difficulty  focusing,  and 
blurred  vision  when  using  the  stereoscopic  glasses.  Only  8.33%  (1/12)  of  the  participants 
experienced  any  problem  when  using  the  HMD  and  only  16.67%  (2/12)  when  using  the 
BOOM.  No  one  experienced  any  symptoms  when  using  the  monoscopic  CRT  monitor 
configuration. 

Additional  Questions 

Hypothesis  #8:  There  is  difference  in  the  how  many  hours  test  participants  are 
willing  to  use  the  four  visual  display  technologies. 

Participants  were  asked  how  many  hours  they  would  be  willing  to  use  each  of  the 
four  visual  display  technologies  (Q27).  A  Kruskal-Wallis  analysis  was  conducted  on  the 
average  time  calculated  from  design  participant  responses.  The  hypothesis  was  that  there 
would  be  no  difference  in  the  average  number  of  hours  that  participants  would  be  willing 
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to  use  the  four  visual  display  technologies.  The  Kruskal-Wallis  analysis  conclude  that 
there  is  significant  evidence  to  reject  the  null  hypothesis  (p<.  03),  indicating  differences 
between  the  four  visual  display  technologies  for  the  amount  of  time  design  participants 
would  be  willing  to  use  the  technologies.  Pairwise  comparison  analysis  revealed  that 
monoscopic  CRT  monitor  and  HMD  systems  differed  significantly  from  the  other 
technologies  and  users  are  willing  to  use  these  technology  systems  for  longer  periods  of 
time. 

The  means  and  standard  deviations  for  each  of  the  visual  display  technologies  are 
provided  in  Table  15  as  a  source  of  insight  into  the  similarities  and  differences  between 
the  four  visual  display  technologies.  Users  are  willing  to  use  the  traditional  monoscopic 
CRT  monitor  for  longer  periods  of  time  compared  to  the  other  three  visual  display 
technologies.  On  average,  they  are  willing  to  use  monoscopic  CRT  monitor  for  5.08 
hours,  HMDs  for  3.33  hours,  and  both  the  BOOM  and  stereoscopic  glasses  for  2.25 
hours. 
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Table  15. 


Average  Time  (Hours)  Subjects  Are  Willing  To  Use  The  Technology 


Technology 

M 

SD 

N 

A 

3.33 

2.42 

12 

B 

2.25 

2.30 

12 

C 

2.25 

2.18 

12 

D 

5.08 

3.20 

12 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 

M  =  mean,  S  =  standard  deviation,  N  =  number. 

A  graph  of  the  least  square  means  of  the  visual  display  technology  vs.  the  time 
willing  to  use  the  technology  dependent  measure  is  provided  in  Figure  19.  This  graph 
supports  previous  ANOVA  findings.  Users  are  willing  to  use  the  traditional  monoscopic 
CRT  monitor  for  longer  periods  of  time  and  are  willing  to  use  the  BOOM  and 
stereoscopic  glasses  for  the  least  amount  of  time. 

Design  participants  were  also  asked  whether  they  thought  the  four  visual  display 
systems  were  more  beneficial  for  individual  use  rather  than  team  use  (Q28).  Design 
participant  responses  are  shown  in  Table  16.  A  Kruskal-Wallis  analysis  was  conducted 
on  the  data.  The  hypothesis  was  that  there  would  be  no  differences  in  whether  the 
participants  felt  the  technologies  were  more  beneficial  for  individual  rather  than  team 
use.  Results  reveal  that  there  is  not  significant  evidence  to  reject  the  null  hypothesis 
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(p=.  83),  indicating  that  no  differences  exist  between  the  four  visual  display  technologies 
for  whether  they  are  more  beneficial  for  individual  use. 


A  b  c  D 
VISUAL  DISPLAY  TECHNOLOGY 

Figure  19.  Least  Square  Mean  Graph  for  Average  Time  (Hours)  Subjects  Are  Willing  To 

Use  The  Technology 

Note:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 


Table  16. 


Frequency  of  Responses  for  Beneficial  for  Individual  Use 


VISUAL  DISPLAY  TECHNOLOGY 

A 

R 

c 

D 

Y 

N 

Y 

N 

Y 

N 

Y 

N 

Q28 

Is  the  technology  more  beneficial  for 
individual  use? 

4 

8 

9 

3 

6 

6 

6 

6 

I 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
"  Y=Yes,  N=No. 
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Additionally,  the  survey  asked  participants  if  there  was  any  aspect  of  the 
visualization  tools  that  could  be  improved  to  enhance  the  CDR  activity  (Q29).  Listings  of 
their  responses  are  included  in  Appendix  N,  participant  comments.  In  general, 
participants  wanted  improved  resolution  of  displays,  better  ability  to  navigate  in  the 
model,  and  the  capability  to  dynamically  change  objects  in  the  virtual  model. 

Preferences:  VE  Assessment  Survey  Three  —  Technology  Comparisons 

Technology  comparisons  were  conducted  to  assess  differences  between  the 
technologies  in  terms  of  their  usefulness,  difficulty,  practicality,  stimulation  of  team 
interactivity,  and  development  of  team  consensus  to  the  CDR  process.  Nonparametric 
Friedman  tests  and  follow-on  multiple  comparison  procedures  as  described  by  Daniel 
(1978)  were  used  to  analyze  the  subjective  comparison  data.  This  test  is  often  used  for 
analyzing  ranks  of  three  or  more  objects  by  multiple  judges.  Individual  participants  were 
asked  to  rank  the  technologies  on  a  scale  of  one  to  four  corresponding  to  a  ranking  from 
best  to  worst.  Where  the  value  of  one  was  considered  the  most  useful,  most  difficult, 
most  practical,  most  helpful  in  stimulating  team  interactivity  or  most  beneficial  for 
developing  team  consensus.  Table  17  provides  the  frequency  of  ranked  data  per 
technology  The  mean  and  standard  deviation  for  each  ranking  of  the  dependent  variable 
per  visual  display  technology  is  provided  in  Table  18. 

Usefulness 

Hypothesis  #9:  There  will  be  no  significant  differences  among  test  participants  in 
their  ratings  of  four  visual  display  technologies  usefulness  during  CDR  task. 
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The  criterion  used  to  assess  the  usefulness  of  the  technologies  was  the  subjective 
ranking  of  the  four  visual  display  technologies.  The  results  of  the  Freidman  tests,  as 
measured  by  individual  rankings,  indicated  that  usefulness  of  the  technologies  on  the 
CDR  process  did  differ  significantly  (p<.  05)  between  the  individual  participants. 

A  multiple  comparison  procedure  was  used  to  determine  the  differences  found  in 
the  Freidman  test.  Results  of  this  test  indicated  that  the  usefulness  of  the  HMD  and  the 
monoscopic  CRT  monitor  were  significantly  different  and  ranked  more  useful  than  the 
BOOM  and  stereoscopic  glasses. 

Figure  20  contains  a  bar  chart,  which  illustrate  these  findings.  Design  team 
participants  felt  that  the  most  useful  technologies  were  the  HMD  and  monoscopic  CRT 
monitor,  while  the  BOOM,  and  stereoscopic  glasses  systems  were  least  useful. 


VISUAL  DISPLAY  TECHNOLOGY 
Figure  20.  Bar  Chart  of  Ranked  Data  for  Usefulness 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Scale  l=most  useful  and  4  =  least  useful. 
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Table  17. 


Frequency  of  Ranked  Data  Responses 


Visual 

Display 

Technology 

R 

a 

n 

k 

Usefulness 

Difficulty 

Practicality 

Team 

Interactivity 

Team 

Consensus 

A 

1 

6 

4 

4 

4 

3 

2 

1 

3 

3 

3 

3 

3 

5 

3 

4 

4 

5 

4 

0 

1 

1 

1 

1 

B 

1 

1 

4 

0 

0 

0 

2 

5 

3 

2 

2 

3 

2 

3 

5 

5 

4 

4 

4 

1 

5 

5 

5 

C 

1 

0 

2 

0 

1 

0 

3 

2 

4 

5 

5 

3 

3 

2 

2 

1 

2 

4 

6 

5 

6 

5 

5 

D 

1 

5 

1 

8 

7 

9 

2 

3 

3 

3 

2 

1 

3 

2 

3 

1 

2 

1 

4 

2 

4 

0 

1 

1 

Note:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
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Table  18. 


Mean  and  Standard  Deviation  for  Ranked  Data  Responses 


Dependent 

Variable 

Visual  Display 
Technology 

M 

SD 

N 

Usefulness 

A 

1.917 

.996 

12 

B 

2.750 

1.055 

12 

C 

3.250 

0.866 

12 

D 

2.083 

1.165 

12 

Difficulty 

A 

2.091 

1.044 

11 

B 

2.091 

1.044 

11 

C 

2.909 

1.221 

11 

D 

2.909 

1.044 

11 

Practicality 

A 

2.167 

1.030 

12 

B 

3.250 

0.754 

12 

C 

3.167 

0.937 

12 

D 

1.417 

0.669 

12 

Design  Team 
Interactivity 

A 

2.167 

1.030 

12 

B 

3.250 

6.754 

12 

C 

2.833 

1.115 

12 

D 

1.750 

1.055 

12 

Design 

Team 

Consensus 

A 

2.333 

0.985 

12 

B 

3.167 

0.835 

12 

C 

3.000 

0.953 

12 

D 

1.500 

1.000 

12 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
M  =  mean,  S  =  standard  deviation,  N  =  number. 
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Difficulty 


Hypothesis  #10:  There  will  be  no  significant  differences  among  test  participants 
in  their  ratings  of  four  visual  display  technologies  difficulty  of  use  during  a  CDR  task. 

The  criterion  used  to  assess  difficulty  of  using  the  technologies  for  a  CDR  was  the 
subjective  ranking  of  the  four  test  systems.  The  results  of  the  Freidman  tests,  as 
measured  by  individual  rankings,  indicated  that  the  difficulty  of  using  the  visual  display 
technologies  during  a  CDR  did  not  significantly  differ  (p>.  20). 

Practicality 

Hypothesis  #11:  There  will  be  no  significant  differences  between  test  participants 
in  their  ratings  of  four  visual  display  technologies  for  practicality  of  use  during  CDR 
task. 

The  criterion  used  to  assess  the  practicality  of  using  the  technologies  for  a  CDR 
was  the  subjective  ranking  of  the  four  visual  display  technologies.  The  results  of  the 
Freidman  tests  as  measured  by  individual  rankings  indicated  that  the  practicality  of  using 
the  four  technologies  during  a  CDR  did  differ  significantly  (p<.  005)  between  the 
individual  participants. 

A  multiple  comparison  procedure  was  used  to  determine  the  differences  found  in 
the  Freidman  test.  Results  of  the  pairwise  comparison  analysis  indicate  that  the 
practical,  ty  of  using  the  monoscopic  CRT  monitor  and  HMD  for  CDR  differed 
significantly  from  the  other  two  visual  display  technologies.  Test  participants  felt  these 
systems  were  the  most  practical  and  BOOM  and  stereoscopic  glasses  were  the  ieast. 
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Figure  21  contains  a  bar  chart,  which  illustrates  these  findings.  Design  team  participants 
felt  that  the  most  practical  technology  was  the  monoscopic  CRT  monitor,  followed  by  the 
HMD,  while  the  least  practical  were  the  stereoscopic  glasses,  and  BOOM  systems. 


VISUAL  DISPLAY  TECHNOLOGY 

Figure  2 1 .  Bar  Chart  of  Ranked  Data  for  Practicality 

Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Scale  l=most  practical  and  4  =  least  practical. 

Team  Interactivity 

Hypothesis  #12:  There  will  be  no  significant  differences  between  test  participants 
in  their  ratings  of  four  visual  display  technologies  for  design  team  interactivity  during 
CDR  task. 

The  criterion  used  to  assess  the  stimulation  of  design  team  interactivity  when 
using  the  technologies  for  a  CDR  was  the  subjective  ranking  of  the  four  visual  display 
technologies,  where  1  was  considered  the  most  stimulating  for  team  interaction  and  4  was 
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the  least  stimulating  for  team  interaction.  The  results  of  the  Freidman  tests,  as  measured 
by  individual  rankings,  indicated  that  the  four  technologies  did  differ  significantly 
(pc.  025)  when  evaluating  their  effect  on  the  stimulation  of  team  activity. 

Pairwise  comparison  analysis  was  used  to  determine  the  differences  found  in  the 
Freidman  test.  Results  of  pairwise  comparison  testing  revealed  that  the  monoscopic  CRT 
monitor  and  HMD  systems  were  the  most  useful  for  team  interaction  and  these  systems 
differed  significantly  from  both  the  BOOM  and  the  stereoscopic  glasses  systems.  Figure 
21  contains  a  bar  chart,  which  illustrate  these  findings.  Design  team  participants  felt  that 
the  technologies  that  assisted  in  design  team  interaction  were  the  monoscopic  CRT 
monitor  and  HMD  systems,  while  the  stereoscopic  glasses,  and  BOOM  systems  were  the 
least  useful  for  design  team  interaction. 
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VISUAL  DISPLAY  TECHNOLOGY 


Figure  22.  Bar  Chart  of  Ranked  Data  for  Design  Team  Interactivity 


Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Scale  l=most  useful  for  team  interactivity  and  4  =  least  useful. 
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Team  Consensus 


Hypothesis  #13:  There  will  be  no  significant  differences  between  test  participants 
in  their  ratings  of  four  visual  display  technologies  for  usefulness  for  developing  design 
team  consensus  during  CDR  task. 

The  criterion  used  to  assess  the  usefulness  for  developing  design  team  consensus 
when  using  the  technologies  in  a  CDR  was  the  subjective  ranking  of  the  four  visual 
display  technologies,  where  1  was  considered  the  most  useful  for  developing  group 
consensus  and  4  was  the  least  useful  for  developing  group  consensus.  The  results  of  the 
Freidman  tests,  as  measured  by  individual  rankings,  indicated  that  the  four  visual  display 
technologies  differ  significantly  (pc.  01)  when  evaluating  their  effect  on  the  usefulness  of 
the  technology  for  developing  team  consensus. 

A  multiple  comparison  procedure  was  used  to  determine  the  differences  found  in 
the  Freidman  test.  Results  of  the  pairwise  comparison  procedure  indicated  that  use  of 
the  monoscopic  CRT  monitor  differed  significantly  from  the  other  technologies  and  was 
ranked  the  best  technology  for  usefulness  in  developing  team  consensus.  Results  also 
revealed  that  the  HMD  system  differed  significantly  from  the  BOOM  and  stereoscopic 
glasses  which,  were  ranked  the  worst.  Figure  23  contains  a  bar  chart,  which  illustrates 
these  findings.  Design  team  participants  felt  that  the  technology  that  best  assisted  in 
design  team  consensus  was  the  monoscopic  CRT  monitor  followed  by  the  HMD,  and 
jointly  by  stereoscopic  glasses,  and  BOOM  systems. 
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VISUAL  DISPLAY  TECHNOLOGY 


Figure  23.  Bar  Chart  of  Ranked  Data  for  Design  Team  Consensus 


Notes:  A=HMD,  B=BOOM,  C=Stereoscopic  Glasses,  D=  Monoscopic  CRT  Monitor. 
Scale  l=most  useful  for  team  consensus  and  4  =  least  useful. 


Summary 

The  results  of  the  experiment  reveal  that  VE  technology  had  an  important  role  in 
the  CDR  process.  The  findings  encourage  further  research  and  development  of  VE  tools 
for  product  design  and  evaluation.  This  experimental  procedure  identified  differences  and 
similarities  between  the  four  visual  display  systems. 

Empirical  evaluation  of  the  CDR  task  measured  the  number  of  design  errors 
detected,  the  average  time  to  detect  a  design  error,  and  the  average  time  to  resolve  a 
design  error.  The  most  number  of  design  errors  detected  were  when  design  teams  used 
either  the  stereoscopic  glasses  or  the  monoscopic  CRT  monitor.  Evaluation  of  the 
average  time  to  detect  a  design  error  concluded  that  design  teams  were  able  to  detect 
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design  errors  faster  when  using  the  HMD.  There  were  no  significant  differences  between 
the  four  visual  display  systems  on  average  resolution  time. 

Test  participants  perceptions  of  each  technology  investigated  in  this  study  were 
assessed  using  data  collected  in  VE  Assessment  Survey  Two.  The  assessment  was 
divided  into  four  main  sections  with  some  additional  questions.  The  first  section 
addressed  whether  the  design  team  participants  felt  they  received  adequate  training  prior 
to  the  experiment.  It  was  determined  that  the  15-minute  training  period  was  sufficient. 
However,  some  users  had  difficulty  understanding  the  functionality  of  the  three-button 
mouse  device  in  the  stereoscopic  glasses  and  monoscopic  CRT  configurations.  The 
second  section  focused  on  the  value  of  the  technologies  on  the  overall  CDR  process. 
Results  showed  that  the  test  participants  believed  that  the  stereoscopic  glasses  had  the 
least  impact  on  the  CDR  process.  In  the  third  section,  test  participant’s  assessed  the 
quality  of  the  CDR  process  when  using  the  test  technologies.  Design  team  participants 
perceived  the  stereoscopic  glasses  with  monitor  system  to  have  the  worst  impact  on 
quality  of  the  CDR  process.  The  fourth  section  investigated  the  physiological  effects 
experienced  by  the  users  after  exposure  to  the  technologies.  The  exposure  time  for  each 
participant  was  too  short  to  assess  these  effects.  However,  some  users  experienced  some 
eye  fatigue  and  discomfort  when  using  the  stereoscopic  glasses. 

Additional  questions  of  the  study,  determined  that  users  were  willing  to  use  the 
monoscopic  CRT  monitor  or  HMD  for  longer  periods  of  time  when  compared  to  the 
other  visual  display  systems.  On  average,  they  were  willing  to  use  the  monoscopic  CRT 
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monitor  for  5.08  hours,  the  HMD  for  3.33  hours,  and  both  the  stereoscopic  glasses  and 
BOOM  for  2.25  hours.  Users  were  also  asked  if  they  thought  that  the  visual  display 
technologies  were  more  beneficial  for  individual  use.  Kruskal-Wallis  test  results 
conclude  that  there  is  no  significant  evidence  to  reject  the  null  hypothesis,  indicating  no 
differences  exist  between  the  four  visual  display  technologies  for  whether  they  are  more 
beneficial  for  individual  use. 

Design  participant  preferences  were  evaluated  from  ranked-data  captured  in  VE 
Assessment  Survey  Three.  Design  team  members  felt  that  the  HMD  and  monoscopic 
CRT  monitor  were  the  most  useful  technologies  to  use  for  CDRs  and  the  most  helpful  for 
team  interaction.  They  felt  that  the  monoscopic  CRT  monitor  was  the  most  practical  and 
most  useful  for  team  consensus  forming.  Test  participants  also  thought  that  all  the  test 
technologies  were  equally  as  difficult  to  use  for  CDR  activities. 

The  next  chapter  will  conclude  the  study  and  provide  guidelines  and 
recommendations  for  VE  applications  to  CDR  process  and  user  interface  design  derived 
from  this  research. 
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CHAPTER  5 


DISSCUSSION  AND  RECOMMENDATIONS 

The  purpose  of  this  study  was  to  evaluate,  through  comparative  empirical  testing 
and  data  analysis,  how  several  commercially -available  virtual  3-D  visual  display  systems 
contribute  to  cross-functional  team  collaboration  in  a  product  design  review 
environment.  These  display  systems  are  3-D  HMD,  BOOM,  Stereoscopic  glasses  and 
monitor  systems,  and  for  comparison  a  traditional  2-D  monoscopic  CRT  monitor. 
Investigation  revealed  significant  differences  in  the  number  of  design  errors  detected,  and 
the  time  to  detect  a  design  error  for  team  performance  during  a  concept  design  evaluation 
using  the  3-D  and  2-D  visualization  technologies.  Empirical  data  also  yielded 
statistically  significant  differences  when  comparing  individual  perception  of  the  value  of 
the  technologies  and  preferences  for  their  use. 

After  summarizing  research  questions  and  findings,  this  chapter  provides  a 
discussion  of  each  topic  area.  Guidelines  for  applying  virtual  visual  display  technologies 
to  the  CDR  environment  and  applications  to  human  interface  design,  and  usability 
engineering  are  discussed  based  on  the  research  results. 

The  Problem.  The  application  of  VEs,  an  emerging  technology,  to  the  CDR 
process  has  intrigued  many  researchers  and  practitioners  in  industry,  government,  and 
academia.  This  technology  creates  a  unique  capability  with  the  potential  to  reduce  cost. 
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shrink  time  to  market,  and  improve  overall  product  quality  by  allowing  multifunctional 
teams  to  make  critical  decisions  early  in  the  design  process.  However,  a  proliferation  of 
visual  display  tools  used  to  generate  3-D  stereoscopic  visual  information,  has  forced 
organizations  to  make  decisions  about  which  tools  to  integrate  to  improve  their  design 
process.  Today,  most  organizations  do  not  have  the  resources  to  conduct  a  comparative 
study  of  these  3-D  visual  display  devices  to  make  appropriate  trade-off  decisions.  As  a 
result,  often  their  decisions  are  arbitrary.  Answers  to  the  following  questions  were  sought 
in  this  study: 

1 .  Number  of  Errors  Detected.  Do  design  teams  detect  and  identify  more  design  errors 
when  using  3-D  visualization  tools? 

2.  Time  to  Detect.  Do  design  teams  detect  and  identify  design  errors  more  quickly  when 
using  3-D  visualization  tools? 

3.  Time  to  Resolve.  Do  design  teams  resolve  design  problems  more  quickly  when  using 
3-D  visualization  tools? 

4.  Perception:  VE  Assessment  Survey  Two  --  the  individual  technology. 

a.  Training.  Was  device  training  prior  to  the  experiment  adequate? 

b.  Design  Review  Process.  What  is  the  perceived  value  of  the  design  process? 

c.  Quality.  Do  design  team  participants  perceive  improved  quality  of  the  design 
process  when  using  the  3-D  display  devices? 

d.  Physiological.  What  is  the  physiological  state  of  the  design  team  members  after 
using  3-D  visualization  tools? 
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e.  Additional  Questions.  How  many  hours  are  they  willing  to  use  the  technologies, 
and  were  the  technologies  more  beneficial  for  individual  use  rather  than  team  use? 

5.  Preferences:  VE  Assessment  Survey  Three  —  technology  comparisons. 

a.  Usefulness.  What  are  preferences  for  usefulness  when  comparing  the  four  display 
technologies? 

b.  Difficulty.  What  are  preferences  for  difficulty  when  comparing  the  four  display 
technologies? 

c.  Practicality.  What  are  preferences  for  practicality  when  comparing  the  four 
display  technologies? 

d.  Team  Interactivity.  What  are  preferences  for  stimulation  for  team  interactivity 
when  comparing  the  four  display  technologies? 

e.  Team  Consensus.  What  are  preferences  for  development  of  team  consensus  when 
comparing  the  four  display  technologies? 

The  Findings.  Results  from  the  investigation  of  study  questions  are  summarized  in 
Table  19.  They  are  as  follows: 

1 .  Number  of  Errors  Detected.  The  null  hypothesis  was  rejected,  indicating  a  significant 
difference  in  the  number  of  design  errors  detected  across  the  four  technology 
treatments.  Pairwise  comparison  testing  revealed  that  design  teams  were  able  to 
detect  more  errors  when  using  either  the  stereoscopic  glasses  or  the  monoscopic  CRT 
monitor  and  the  least  when  using  the  HMD  or  BOOM  systems. 
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Table  19. 


Summary  of  Test  Results 


Research  Question 

Best 

Worst 

1 

Errors  detected 

Stereoscopic  Glasses  & 
Monoscopic  CRT 

Monitor 

HMD  &  BOOM 

2 

Time  to  detect 

HMD 

Stereoscopic  Glasses  &  BOOM 

3 

Time  to  resolve 

Not  Statisucally  Significant 

4 

Perception 

a. 

Technology  Training 

Not  Statistically  Significant 

b. 

Design  Review  Process 

HMD  &  BOOM 

Stereoscopic  Glasses 

c. 

Quality 

HMD  &  Monoscopic 

CRT  Monitor 

Stereoscopic  Glasses 

d. 

Physiological 

Not  Statistically  Significant 

e. 

Additional  Questions 

Number  of  Hours 

Willing  to  Use  the 
Technology 

Monoscopic  CRT 

Monitor  &  HMD 

Stereoscopic  Glasses 

Beneficial  for  Individual 
Use 

Not  Staustically  Significant 

5 

Preferences 

a. 

Usefulness 

HMD  &  Monoscopic 

CRT  Monitor 

Stereoscopic  Glasses  &  BOOM 

b. 

Difficulty 

Not  Statisucally  Significant 

c. 

Practicality 

Monoscopic  CRT 

Monitor 

Stereoscopic  Glasses  &  BOOM 

d. 

Team  Interactivity 

Monoscopic  CRT 

Monitor  &  HMD 

Stereoscopic  Glasses  &  BOOM 

e. 

Team  Consensus 

Monoscopic  CRT 

Monitor 

Stereoscopic  Glasses  &  BOOM 
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2.  Time  to  Detect.  The  null  hypothesis  was  rejected,  indicating  a  significant  difference 
in  the  average  time  to  detect  a  design  error  across  the  four  treatment  technologies. 
Post  hoc  comparison  analysis  concluded  that  design  teams  were  able  to  detect  errors 
faster  when  using  the  HMD  system. 

3.  Time  to  Resolve.  The  null  hypothesis  was  retained  indicating  that  there  were  no 
significant  differences  among  the  four  treatment  visual  display  technologies  in  the 
average  time  to  resolve  a  detected  design  problem.  Outside  observation  revealed  that 
this  was  due  mainly  to  the  fact  that  design  team  members  elected  not  to  use  any 
technologies  during  this  phase  of  the  CDR  activity.  Team  members  chose  to  resolve 
design  problems  using  natural,  face-to-face  communication. 

4.  Perception:  VE  Assessment  Survey  Two  —  the  individual  technology. 

a.  Trainine.  Based  on  descriptive  analyses  of  participant  perceptions  of  the  15- 
minute  training  period,  it  appeared  that  they  received  an  adequate  amount  of  training 
prior  to  conducting  the  experiment. 

b.  Design  Review  Process.  Kruskal-Wallis  analysis  reveal  significant  differences 
between  the  four  visual  display  systems  for  the  value  on  the  CDR  process  dependent 
variable.  Pairwise  comparison  testing  concluded  that  the  HMD,  BOOM,  and  monoscopic 
CRT  monitor  systems  differed  significantly  from  the  stereoscopic  glasses.  Participants 
felt  these  systems  had  positive  effect  on  the  design  review  process. 

c.  Quality.  Kruskal-Wallis  analysis  reveal  significant  differences  between  the  four 
visual  display  systems  for  the  value  on  the  quality  dependent  variable.  Pairwise 


172 


comparison  testing  indicate  that  the  HMD,  BOOM,  and  monoscopic  CRT  monitor 
systems  were  significantly  different  and  provided  better  quality  to  the  CDR  process  than 
stereoscopic  glasses. 

d.  Phvsioloeical.  The  physiological  state  of  the  design  team  members  after  using  the 
3-D  visualization  tools  could  not  be  sufficiently  assessed  because  the  exposure  time  was 
too  short.  However,  users  experienced  some  discomfort  after  exposure  to  the 
stereoscopic  glasses. 

e.  Additional  Questions.  Kruskal-Wallis  analyses  on  the  average  time  users  were 
willing  to  use  the  visual  display  systems  revealed  significant  differences  between  the  four 
visual  display  devices.  Pairwise  comparison  testing  indicated  that  the  stereoscopic  glasses 
differed  significantly  and  performed  the  worst  when  comparing  how  many  hours  test 
participants  were  willing  to  use  the  technologies.  Descriptive  statistical  analysis  revealed 
that  design  team  participants  are  willing  to  use  a  traditional  monoscopic  monitor  34% 
longer  than  a  HMD  and  55%  longer  than  either  the  BOOM  or  stereoscopic  glasses. 

Additionally,  Kruskal-Wallis  analysis  was  also  conducted  on  the  data  from  the 
question  whether  users  felt  that  the  technologies  were  better  suited  for  individual  rather 
than  team  use.  This  test  revealed  that  there  was  not  sufficient  evidence  to  detect 
differences  between  the  four  visual  display  systems. 

5.  Preferences:  VE  Assessment  Survey  Three  —  technology  comparisons. 

a.  Usefulness.  Results  of  Freidman  analysis  concluded  that  the  null  hypothesis  was 
rejected,  indicating  differences  in  design  participant’s  preference  for  usefulness. 
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Pairwise  comparison  testing  indicate  that  the  HMD  and  monoscopic  CRT  monitor 
were  significantly  different  and  more  useful  than  the  BOOM  and  stereoscopic  glasses 
systems. 

b.  Difficulty.  The  null  hypothesis  for  individual  preference  for  difficulty  was 
retained  indicating  no  significant  differences  in  the  design  participant’s  perception  of 
the  difficulty  of  using  the  four  visual  display  systems. 

c.  Practicality.  Results  of  Freidman  analysis  concluded  that  the  null  hypothesis  was 
rejected,  indicating  differences  in  design  participant’s  preference  for  practicality. 
Post-hoc  pairwise  comparison  testing  revealed  that  the  monoscopic  CRT  monitor 
differed  significantly  and  was  considered  the  most  pracucal  technology.  Users  felt 
that  both  the  stereoscopic  glasses  and  BOOM  were  the  least  practical. 

d.  Team  Interactivity.  Results  of  Freidman  analysis  concluded  that  the  null 
hypothesis  was  rejected,  indicating  differences  in  design  participant’s  preference  for 
design  team  interactivity  dependent  variable.  Pairwise  comparison  testing  indicated 
that  the  monoscopic  CRT  monitor  and  HMD  systems  were  the  most  helpful 
technology  to  stimulate  team  interaction  and  differed  significantly  from  both  the 
BOOM  and  stereoscopic  glasses  systems. 

e.  Team  Consensus.  Freidman  analysis  concluded  rejection  of  the  null  hypothesis, 
indicating  differences  in  design  participant’s  preference  for  helping  develop  team 
consensus.  Follow-on  pairwise  testing  revealed  that  design  participants  felt  that  the 
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f.  monoscopic  CRT  monitor  best  helped  team  consensus  formulating  followed  by 
the  HMD,  and  jointly  by  the  stereoscopic  glasses  and  BOOM  systems. 

Discussion 

The  discussion  addresses  the  findings  with  regard  to  number  of  errors  detected, 
time  to  detect,  time  to  resolve,  perception,  and  preferences. 

Number  of  Design  Errors  Detected 

Results  indicated  that  significant  differences  exist  between  the  four  visual  display 
technologies  in  the  number  of  errors  detected  during  a  CDR.  This  can  be  interpreted  to 
mean  that  the  number  of  errors  detected  when  conducting  a  CDR  and  evaluation  task  are 
dependent  on  which  visual  display  technology  is  used.  The  results  clearly  show  that  a 
greater  number  of  design  errors  were  detected  when  design  teams  used  the  stereoscopic 
glasses  or  the  monoscopic  CRT  monitor  systems.  Post  hoc  analysis  indicated  that  these 
systems  performed  similarly  and  differed  significantly  from  the  HMD  and  BOOM 
systems,  which  preformed  the  worst. 

This  implies  that  team  interaction  is  critical  in  this  type  of  task.  This  finding 
supports  the  basic  underlying  premise  of  concurrent  engineering.  As  discussed  by  Clark 
and  Fujimoto  (1991),  informal  face-to-face  relationships  between  functional 
organizations  is  an  important  necessity  in  any  organization.  This  also  supports  the  fact 
that  person-to-person  communication  is  a  key  activity  in-group  work  (Floyd  &.  Turner, 
1989).  The  need  for  multi-functional  design  teams  to  discuss  and  interact  when 
conducting  a  design  review  is  fundamental  for  the  success  of  the  design  review.  Both  the 
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stereoscopic  glasses  and  monoscopic  CRT  monitor  configurations  allowed  design  teams 
to  simultaneously  interact  with  the  virtual  model.  In  these  two  configurations,  all 
participants  were  able  to  see  the  same  model  view,  concurrently.  This  created  a  more 
natural,  intuitive  communication  mechanism  between  team  members.  Outside 
observation  indicated  that  when  using  a  common  model  view,  participants  appeared  to  be 
comfortable  as  if  they  were  in  a  more  familiar  environment.  There  was  no  need  for  team 
members  to  switch  devices  or  places  to  view  the  model  --  activities  that  can  distract  team 
activity.  It  also  implies  that  in  team  settings  it  is  critical  that  persons  see  the  same  virtual 
model  simultaneously.  When  all  participants  were  able  to  see  the  3-D  stereoscopic  model 
they  were  able  to  discuss  problems  through  team  interaction  and  this  promoted  the 
identification  of  new  design  errors.  This  implies  that  for  team  activity  tasks  such  as 
CDRs,  VE  tools  need  to  maintain  the  feeling  of  team  cohesiveness. 

In  contrast,  inherent  properties  of  the  HMD  and  BOOM  configurations  prevented 
the  entire  design  team  to  simultaneously  interact  and  view  the  TFTS  model.  Allowing 
only  one  person  to  control  the  model  view  restricted  team  dialogue  and  interaction  and 
yielded  lower  performances  in  the  number  of  errors  detected.  Supporting  this  claim,  one 
participant  commented  “the  guy  in  the  HMD  is  like  in  the  closet,”  and  another  said  “the 
BOOM  user  is  the  odd  man  out  and  this  is  a  problem  with  this  technology.” 

In  addition,  results  from  this  study  reflect  similar  findings  to  those  concluded  by 
McWhorter,  Hodges,  and  Rodriguez  (1991).  Their  work  focused  on  ranking  different 
types  of  display  formats  common  to  CAD  applications  for  geometric  information 
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conveyed  and  perceived  realism  of  objects.  They  found  that  stereo  displays  were  judged 
superior  to  non-stereo  displays  in  providing  users  with  geometric  information  about  an 
object.  It  is  apparent  that  the  effect  of  stereoscopic  vision  is  important  in  better 
understanding  the  data  and  for  easier  recognition  of  form  and  pattern. 

An  interesting  point  that  can  be  made  which  is  based  on  user  perceptions  and  this 
researcher’s  observations  is  that  users  did  not  prefer  to  use  the  stereoscopic  glasses,  yet 
design  teams  performed  better  when  using  this  technology.  Most  of  the  comments  and 
observations  about  the  stereoscopic  glasses  were  negative.  One  user  said  “glasses  are 
hindersome  at  times”  and  another  user  commented,  “developed  a  bit  of  an  eyestrain 
trying  to  focus.”  From  an  interface  usability  standpoint  it  can  be  inferred  that  3-D 
stereoscopic  visual  perception  of  objects  and  common  viewing  of  objects  are  important  in 
helping  design  teams  find  and  detect  design  errors. 

The  four-way  ANOVA  also  detected  differences  between  the  design  teams.  The 
obvious  explanation  is  that  even  though  design  teams  were  created  with  similar 
characteristics  in  terms  of  professional  experience,  academic  backgrounds,  computer 
skills  there  still  was  one  major  factor  that  could  not  be  controlled.  That  is  human 
individualism  and  varying  approaches  to  problem  solving.  It  was  evident  that  during  the 
experiment  there  were  different  types  of  people  with  different  logic  approaches  to 
problem  solving.  Researcher  observation  reflected  that  the  approach  a  team  took  to 
problem  solving  was  based  on  the  individual  personality  types  of  the  team  members. 
These  variation  in  personalities  and  people  ranged  from  strong  and  boisterous  to  non- 
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aggressive  and  shy.  These  results  support  the  beliefs  of  Shneiderman  (1993)  that 
personality  type  plays  an  important  role  in  designing  systems.  He  stressed  that  a  clear 
understanding  of  personality  and  cognitive  styles  for  specific  communities  of  users  is 
critical  when  designing  interactive  systems. 

Time  to  Detect 

The  results  concluded  that  differences  between  the  four  visual  display 
technologies  exists  in  the  average  time  to  detect  a  design  error  during  a  CDR.  The  results 
reveal  that  the  average  time  to  detect  a  design  problem  or  to  identify  a  design  flaw  when 
using  the  HMD  were  significantly  different  from  the  average  detection  times  when  design 
teams  used  any  of  the  other  three  technologies.  Design  teams  were  able  to  detect  design 
errors  faster  when  using  the  HMD  system.  These  results  may  be  attributable  to  the  effects 
of  immersion  that  the  visual  display  device  creates.  Stuart  (1996)  defines  immersion  as 
the  presentation  of  sensory  cues  that  convey  perceptually  to  users  that  they  are 
surrounded  by  the  computer-generated  environment.  Visual  display  devices  can  be 
categorized  and  defined  by  the  level  of  immersion  they  provide  to  the  user.  These  are 
immersive,  semi-immersive,  and  non-immersive. 

The  HMD  device  creates  an  immersive  environment  where  the  user  experiences 
the  virtual  world  and  is  not  cognizant  of  the  real  world.  This  level  of  immersion  provides 
the  user  with  a  maximum  level  of  presence  in  the  environment.  Hendrix  and  Barfield 
(1996)  define  one  aspect  of  presence  in  a  VE  as  the  sense  of  “being  there.”  The  effect  of 
a  person  being  immersed  in  the  virtual  design  may  have  some  value  to  the  CDR  process. 
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The  downside  of  using  a  single  HMD  system  for  team  activity  is  that  not  all  participants 
are  able  to  share  the  same  experience.  Current  technology  is  capable  of  networking 
multiple  persons  in  a  VE,  where  all  the  users  could  wear  HMDs  and  achieve  this  greater 
level  of  immersion.  Researchers  are  developing  the  infrastructure  to  accomplish  this  goal 
by  improving  communication  protocols,  creating  database  management  procedures,  and 
computer  operating  systems. 

In  contrast,  the  BOOM  and  stereoscopic  glasses  are  considered  semi-immersive 
systems,  thus  creating  an  environment  where  the  user  sees  and  experiences  the  virtual 
world  but  is  cognizant  of  the  real  world.  A  user  can  easily  remove  the  device  and  can  use 
the  real  world  as  a  reference.  The  findings  conclude  that  the  BOOM  and  stereoscopic 
glasses  perform  similarly  and  differed  significantly  from  the  performance  when  teams 
used  the  non-immersive,  monoscopic  CRT  monitor.  Design  team  performed  the  worst 
when  using  the  BOOM  and  stereoscopic  glasses  systems. 

One  conclusion  drawn  from  these  findings  is  that  users  prefer  the  sense  of  total 
presence  in  the  environment  and  if  they  can  not  obtain  that  level  of  immersion  they  prefer 
to  revert  back  to  their  more  familiar  technology,  the  monoscopic  CRT  monitor. 

The  experimental  design  maintained  equivalence  between  design  teams  and 
between  TFTS  subassemblies,  and  the  design  maximized  randomization  of  the 
experiment  order,  yet  the  four-way  ANOVA  yielded  significant  statistical  differences 
between  all  three  blocking  factors.  Differences  in  the  performance  of  the  design  teams 
can  be  attributed  to  the  inability  to  control  variation  in  personality  types  and  logic 
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methods  individuals  use  to  detect  problems.  The  TFTS  subassemblies  chosen  for  this 
study  contained  similar  design  problems  based  on  an  evaluation  from  a  senior  level 
designer.  However,  differences  where  identified  between  the  TFTS  subassemblies.  This 
finding  can  be  attributed  to  the  navigation  required  for  a  design  team  to  get  to  a 
subassembly  location.  Least  mean  square  graphs  for  the  subassembly  factor  shows  that 
the  average  time  to  detect  a  design  error  was  longer  for  the  TFTS  suspension 
subassembly  than  the  other  three  subassemblies.  One  obvious  interpretation  of  this 
finding  is  that  evaluating  the  suspension  system  required  design  teams  to  navigate  to  the 
underside  of  the  TFTS  model  so  that  they  had  the  correct  view  of  the  suspension  system. 
This  maneuvering  could  account  for  the  differences  experienced  between  the 
subassemblies.  The  least  mean  square  graph  for  the  experiment  order  factor  show  that 
design  teams  preformed  worst  for  the  third  experiment  in  their  testing  sequence.  One 
possible  cause  for  this  finding  is  that  design  teams  were  given  a  short  break  at  the  end  of 
their  second  treatment,  which  could  have  resulted  in  a  break  in  continuity. 

Time  to  Resolve 

No  significant  differences  were  revealed  for  the  average  time  to  resolve  a  detected 
design  problem.  This  study  failed  to  find  any  significance  between  the  four  visual 
display  technologies  on  the  time  it  took  the  design  teams  to  resolve  a  detected  design 
problem. 

One  possible  reason  is  that  after  a  problem  was  identified  the  design  teams  solved 
the  problem  without  the  assistance  of  any  of  the  four  visual  display  systems.  Team 
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members  reverted  back  to  basic  communication  skills  and  practices,  at  this  stage  of  the 
CDR  activities.  It  is  apparent  that  problem  resolution  within  teams  requires  face  to  face 
communications.  The  value  of  human  interaction  through  gestures,  eye  contact,  and 
mannerisms  is  crucial  in  consensus  forming  tasks.  Outside  observation  concluded  that 
after  a  period  of  time  the  design  team  members  did  not  use  the  technology  as  the  focus  of 
their  attention.  Instead  they  continued  discussions  to  resolve  the  problem  without  much 
emphasis  on  viewing  the  model. 

In  addition,  some  team  members  who  were  using  stereoscopic  glasses  removed 
them  while  continuing  to  have  team  discussions.  This  action  demonstrated  a  lack  of  need 
to  use  the  technology  during  team  discussion  activities.  At  the  extreme,  one  user  who 
was  not  wearing  stereoscopic  glasses  explained  his  point  of  view  by  pointing  at  the  model 
on  the  monitor  even  though  the  image  was  blurred  and  distorted.  This  action  support 
earlier  research  which  identifies  a  users  need  for  face-to-face  communication  to  include 
gesture  and  expression  when  discussing  and  solving  design  problems. 

Perception:  VE  Assessment  Survey  Two  —  The  Individual  Technology 

VE  Assessment  Survey  Two  was  designed  to  obtain  design  team  participant’s 
subjective  feelings  on  the  value  and  usefulness  of  each  of  the  four  visual  display 
technologies.  The  questionnaire  addressed  four  areas:  technology-training,  CDR  process, 
quality,  and  physiological  effects.  Some  additional  questions  were  also  included  to 
determine  how  long  users  were  willing  to  use  the  technologies  and  if  they  thought  the 
systems  were  better  adapted  for  individual  use. 
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Technology  Training 


The  results  indicate  that  the  standard  15  minute  training  period  was  adequate  for 
the  design  teams  to  conduct  the  design  review  task.  Subjective  data  indicate  that  design 
team  participants  received  a  sufficient  level  of  training  prior  to  the  experimentation. 

Based  on  frequency  of  responses,  87.5%  “found  the  training  to  be  sufficient”,  85.4% 
“thought  that  they  fully  understood  the  correct  operating  procedures,  and  only  20.8% 

“felt  they  were  not  prepared.”  Most  negative  responses  were  associated  with  the 
stereoscopic  glasses  and  monoscopic  CRT  monitor  systems.  This  can  be  related  to  the 
navigation  method  employed  in  these  two  configurations.  The  mouse  device  was  more 
difficult  to  learn  and  operate  because  it  required  more  finesse.  From  this  finding,  it  can  be 
concluded  that  caution  must  be  made  when  integrating  simple  mouse  devices  as 
navigation  tools  for  virtual  environment-based  CDRs.  More  training  may  be  required  in 
order  for  users  to  grasp  the  functionality  of  panning  in  the  environment,  a  task  easily 
accomplished  in  the  real  world  with  normal  head  movements.  It  is  important  to 
understand  that  novice  users  get  frustrated  easily  and  can  experience  anxiety  when  using 
new  computer  systems  (Shneiderman,  1993). 

Concept  Design  Review  Process 

Results  from  the  Kruskal-Wallis  analysis  and  Bonferroni  pairwise  comparison 
tests  on  the  cumulated  CDR  process  data  reveal  differences  between  the  four  test  visual 
display  systems.  User  perceptions  indicate  that  design  team  participants  felt  that 
stereoscopic  glasses  have  the  least  effect  on  the  design  process.  It  is  supported  by 
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descriptive  analyses  performed  on  the  data.  Only  1 6%  (2/12)  felt  that  it  “assisted  the 
design  process”,  only  58%  (7/12)  thought  it  “helped  describe  problems  to  other  team 
members,”  and  only  41%  (5/12)  felt  it  “assisted  in  team  consensus  building.”  In  contrast, 
users  felt  that  the  HMD,  BOOM,  and  monoscopic  CRT  monitor  were  more  helpful  and 
useful  to  the  CDR  process.  Eighty-three  percent  (10/12)  of  the  users  felt  that  the  HMD 
and  BOOM  helped  them  identify  problems.  Caution  needs  to  be  used  when  interpreting 
these  results.  There  is  a  much  promoting,  and  hype  about  VR  technology  in  the  media, 
and  this  was  the  first  exposure  to  these  devices  for  some  of  the  test  participants.  Novice 
users  may  have  been  more  intrigued  with  the  more  novel  visual  VR  display  technologies 
and  felt  that  they  have  a  better  effect  on  the  CDR  process. 

Quality 

Results  from  the  Kruskal -Wallis  analysis  and  Bonferroni  pairwise  comparison 
tests  on  the  cumulated  quality  data  reveal  differences  between  the  four  test  visual  display 
systems.  Participants  felt  that  the  stereoscopic  glasses  had  the  least  effect  on  the  quality 
of  the  CDR  process.  This  is  supported  by  the  descriptive  analysis  findings.  Where  user 
perceptions  indicate  that  only  33.3%  (4/12)  of  the  participants  felt  that  stereoscopic 
glasses  impacted  the  quality  of  the  design  process  in  a  positive  way,  only  16.67%  (2/12) 
thought  that  using  stereoscopic  glasses  were  satisfying  and  wonderful,  and  only  25% 
(3/12)  thought  they  were  stimulating.  This  finding  can  be  attributed  to  the  fact  that  users 
seemed  uncomfortable  in  wearing  the  stereoscopic  glasses.  One  commented  that  “  they 
wouldn’t  want  to  use  the  technology  for  a  long  period  of  time.”  Again  caution  needs  to  be 
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used  when  interpreting  these  results.  Novice  users  may  think  the  more  complex  systems, 
the  HMD  and  BOOM,  are  better  merely  because  of  their  higher  cost,  larger  size,  and 
promotion.  This  study  did  not  explore  this  issue  further. 

Physiological  Effects 

The  information  collected  in  this  section  of  the  questionnaire  addressed  the 
physiological  state  of  the  users  after  exposure  to  the  four  visual  display  systems.  The 
actual  duration  of  usage  for  each  participant  for  each  technology  was  relatively  short 
(approximately  three  to  four  minutes);  therefore,  a  comprehensive  evaluation  of  the 
mental  and  physical  state  of  the  design  team  participants  was  not  feasible.  However, 
descriptive  analyses  indicate  that  the  majority  of  design  team  participants  did  not 
experience  any  simulator  sickness  symptoms.  Only  four  of  the  design  participants 
experienced  any  symptoms  after  exposure  to  the  visual  display  systems.  Of  the  affected 
participants  only  one  person  experienced  any  symptoms  using  the  HMD,  two  using  the 
BOOM,  and  zero  using  the  monoscopic  CRT  monitor.  However,  thirty-three  percent  of 
the  participants  felt  some  form  of  eyestrain  when  using  the  stereoscopic  glasses.  A  more 
in-depth  study  is  required  to  determine  the  physiological  effect  of  using  these  systems  for 
CDR  processes.  These  studies  could  focus  on  determining  the  optimal  exposure  time  for 
these  technologies  applying  simulator  sickness  assessment  techniques  as  described  by 
Kennedy  et  al.  (1992). 
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Additional  Questions 


Individual  design  participants  were  asked  how  many  hours  they  would  be  willing 
to  use  each  of  the  four  test  visual  display  technologies.  Descriptive  statistics  concluded 
that  on  average  users  are  willing  to  use  a  traditional  monoscopic  monitor  34%  longer  than 
a  HMD  and  55%  longer  than  either  the  BOOM  or  stereoscopic  glasses.  User  comments 
support  this  finding.  Some  of  these  comments  were:  “it  was  familiar  as  using  a  PC,” 
“easy  to  use,”  and  “less  tasking  on  one  person."  This  finding  identifies  a  typical 
interface  design  problem  faced  by  new  technology  developers.  New  interface  paradigms 
must  compete  with  standard  computer  interfaces.  According  to  Nielsen  (1993)  new 
interfaces  need  to  be  intuitive  to  users  to  provide  them  with  consistency  especially  if 
these  new  interfaces  take  the  place  of  existing  ones.  The  researcher  defines  an  intuitive 
interface  as  one  that  is  easy  to  learn  and  easy  to  remember,  and  consistency  can  enhance 
the  user’s  ability  to  transfer  a  skill  from  one  system  to  another  or  from  one  task  to 
another.  This  is  a  logical  rationale  to  explain  why  users  have  a  tendency  to  revert  back  to 
what  they  know  —  more  familiar  ground.  If  VEs  are  to  become  the  next  generation 
interactive  interface,  it  will  be  necessary  to  closely  match  functionality  with  user 
expectations. 

Participants  were  also  asked  whether  they  thought  the  visual  display  systems  were 
more  beneficial  for  single  use  rather  than  team  work.  Statistical  analysis  revealed  that  no 
significant  differences  exist  between  the  four  tested  visual  display  systems  with  regard  to 
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this  issue.  There  was  no  evidence  that  the  four  tested  technologies  were  more  useful  for 
single  person  usage. 

Preferences:  VE  Assessment  Survey  Three  —  Technology  Comparisons 

Usefulness 

Results  from  the  Freidman  analysis  indicate  that  individual  preference  for 
usefulness  of  the  technologies  differed  significandy  between  the  four  visual  display 
technologies.  Post-hoc  pairwise  comparison  tests  revealed  that  when  participants  were 
asked  to  compare  the  four  visual  display  technologies,  they  felt  that  the  usefulness  of  the 
HMD  and  the  monoscopic  CRT  monitor  were  equally  useful  and  better  than  both  the 
BOOM  and  stereoscopic  glasses  systems.  Outside  observation  and  user  comments 
support  these  findings.  HMD  comments  include:  “Seems  good  because  it  raised  many 
questions  regarding  the  concept  in  a  short  time,”  and  “you  are  able  to  see  3-D  and  can 
really  concentrate  on  the  design.”  Monoscopic  CRT  monitor  comments  include:  “less 
tasking  on  one  person,  allowing  the  group  to  interact  more,”  “provides  a  better 
understanding  of  the  concept,”  and  “it  was  as  familiar  as  using  your  PC.”  In  contrast, 
user  comments  about  the  BOOM  were  more  negative  and  include:  “device  doesn’t  lend 
itself  to  office  use,”  and  “equipment  was  cumbersome  and  difficult  to  maneuver.” 
Stereoscopic  glasses  comments  include:  “glasses  didn’t  seem  to  make  much  of  a 
difference  or  assist  in  the  design  process,”  “glasses  are  hindersome,”  and  “wouldn’t 
want  to  use  this  technology  for  a  long  design  review.” 
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Difficulty 


When  individual  team  members  were  asked  to  compare  how  difficult  the 
technology  was  when  applied  to  the  CDR  process,  the  Freidman  analysis  concluded  no 
significant  statistical  differences  between  the  four  visual  display  technologies.  All  four 
of  the  evaluated  technologies  gave  the  users  some  difficulty  in  operating  the  system. 

Users  experienced  problems  with  using  all  of  the  navigational  devices:  the  mouse,  the  3- 
D  pointer,  and  the  BOOM  functional  buttons.  User  comments  include:  “difficult  to  use 
the  mouse  to  maneuver  to  a  desired  angle”,  “difficult  to  move  vertically  with  the 
BOOM”,  and  “when  using  the  HMD  locking  in  on  a  single  view  is  hard.” 

Practicality 

The  Freidman  analysis  results  reveal  significant  differences  for  how  practical  the 
technology  is  for  CDR.  Pairwise  comparison  results  indicated  that  practicality  of  using 
the  monoscopic  CRT  monitor  differed  from  all  other  evaluated  technologies.  Users 
believed  that  this  system  was  the  most  practical  and  that  the  BOOM  and  stereoscopic 
glasses  were  the  least  practical  technology  to  use.  These  findings  are  supported  by  user 
comments.  One  person  commented  on  the  BOOM  saying  “  I  could  not  use  this  on  a  daily 
basis.”  Another  said  “Wouldn’t  want  to  use  this  for  long  reviews”  when  talking  abou'  the 
stereoscopic  glasses.  These  findings  demonstrate  that  immaturity  of  the  technologies  may 
have  a  negative  impact  on  real  world  applications.  Comment  analysis  leads  to  the 
conclusion  that  users  realize  that  the  monoscopic  CRT  monitor  is  the  most  practical 
device  because  it  is  the  most  mature  and  familiar. 
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Team  Interactivity 


Results  from  Freidman  analysis  indicated  that  individual  preferences  for  how  the 
technology  assisted  in  stimulating  team  interactivity  between  the  technologies  differed 
significantly  between  the  four  visual  display  technologies.  It  was  found  that  the  HMD 
and  the  monoscopic  CRT  monitor  were  preferred  similarly  and  were  ranked  higher  for 
helping  stimulate  team  interaction.  Users  felt  that  the  BOOM  and  stereoscopic  glasses 
systems  performed  similarly  and  were  ranked  the  worst.  Even  though  the  HMD  system 
did  not  allow  all  the  test  participants  to  use  the  virtual  display  device  simultaneously, 
users  still  felt  that  this  technology  helped  in  team  interaction. 

Team  Consensus 

Design  team  participants  were  asked  to  compare  how  useful  they  felt  the 
technologies  were  in  developing  design  team  consensus.  Friedman  analysis  results 
indicated  that  significant  differences  exist  between  the  four  visual  display  technologies. 

Results  from  the  pairwise  comparison  analysis  show  that  the  monoscopic  CRT 
monitor  differed  from  the  other  three  visual  display  technologies  and  that  the  BOOM  and 
stereoscopic  glasses  were  similar.  Design  team  participants  thought  that  the  monoscopic 
CRT  monitor  was  the  best  for  helping  design  teams  form  consensus  and  that  the 
stereoscopic  glasses  and  BOOM  systems  were  the  worst.  This  finding  indicates  that  the 
CRT  monitor  configuration  was  better  suited  for  team  communication.  When  all  team 
members  were  able  to  simultaneously  view  the  same  perspective  of  the  TFTS  model  team 
consensus  building  was  enhanced. 
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Recommendations 


Machover  (1997)  provides  worldwide  forecasts  for  the  year  2000  for  commercial 
and  industrial  computer  graphics  applications  and  are  shown  in  Table  20.  These 
projections  demonstrate  expanded  growth  and  strong  emphasis  on  computer  display 
systems.  As  shown,  CAD/CAM,  while  small  compared  to  other  technologies,  will  grow 
22%  and  VR  applications  will  grow  81%  by  the  year  2000.  This  is  the  largest  projected 
growth  of  all  computer  applications. 

Table  20. 

Worldwide  Forecasts  for  Commercial  and  Industrial  Computer  Graphics  Applications 


Application 

1995 

2000 

Percent  Growth 

(Sbillion) 

($biilion) 

(forecasted) 

CAD/CAM 

$11.9 

$15.2 

22 

Art/animation 

$3.4 

$7.4 

54 

Multimedia  Presentation 

$14.7 

$29.5 

50 

Real-time  Simulation 

$0.7 

$  1.3 

46 

Scientific  Visualization 

$  2.9 

$6.5 

55 

Graphic  Arts 

$4.0 

$11.2 

64 

Virtual  Reality 

$0.4 

$2.1 

81 

Other 

$5.3 

$  8.7 

39 

Total 

$43.3 

$81.9 

47 

Note:  Modified  from  How  Applications  Have  Driven  Display  Requirements  (p.  4), 
by  C.  Machover.  In  L.  W.  MacDonald  &  A.  C.  Lowe  (Ed.)  Display  Systems: 
Design  and  Applications.  Chichester,  UK:  John  Wiley  &  Sons;  1997. 


In  light  of  these  projections,  research  is  needed  to  gain  a  better  understanding  of 
how  to  apply  display  technologies  to  these  application  areas  while  maintaining  and 
improving  performance  of  the  users.  Based  on  the  results  of  the  research  study,  this 
section  offers  recommendations  for  application  to  CDRs  and  user  interface  design  and 
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usability  engineering  practices.  This  section  also  presents  recommendations  for  future 
studies  in  these  two  areas. 

Application  to  Concept  Design  Reviews 

This  study  has  shown  that  VE  technology  has  a  potential  to  reduce  development 
costs  by  identifying  design  problems  early  in  the  design  process  through  the  ability  of 
cross-functional  design  teams  simultaneously  interacting  in  virtual  prototypes  of  future 
designs.  This  section  reflects  on  the  results  of  this  study  and  offers  some 
recommendations  for  design  trade-off,  software  issues,  and  dynamic  modeling  —  a 
concept  referred  to  as  VE-CAD. 

Today,  wide  ranges  of  visual  display  devices  and  VE  tools  are  available  in  the 
market.  Similarly,  cost  of  these  systems  varies  widely.  The  risk  organizations  face  is  they 
must  make  decisions  on  which  VE  tools  to  invest  in  and  which  tools  are  most  appropriate 
for  the  tasks  they  wish  to  accomplish.  This  requires  organizations  to  make  design  trade¬ 
offs  based  on  the  state  of  the  current  technology,  the  task  to  be  accomplished,  demands 
on  human  perceptual  and  motor  capabilities,  and  cost. 

Based  on  empirical  findings,  it  can  be  inferred  that  the  best  approach  to  using  the 
commercially  available  visual  display  systems  tested  in  this  study  for  CDR  activities  is  a 
combined  technology  approach.  Before  the  CDR  process  begins  using  virtual  visual 
display  technologies,  design  team  members  should  clearly  understand  the  objectives  of 
the  CDR  and  the  requirement  specification  of  the  system  they  are  evaluadng.  Once  team 
members  possess  this  understanding,  the  CDR  process  can  be  initiated  using  a  combined 
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technology  approach.  The  following  recommendation  is  made  based  on  the  findings  of 
this  study. 

First,  using  an  HMD  system,  allow  design  team  participants  to  review  the 
conceptual  virtual  model  individually.  This  complies  with  the  second  empirical  finding, 
that  design  teams  were  able  to  identify  design  errors  faster  with  this  system.  When 
selecting  a  HMD  system  it  is  important  to  purchase  a  system  that  has  a  higher  resolution, 
greater  FOV,  and  less  physical  weight  because  these  three  technical  features  help 
improve  the  sense  of  immersion  in  the  virtual  world  and  minimize  ergonomic  problems. 
Normally  these  systems  are  more  costly  because  of  the  more  sophisticated  optics. 

After  individual  evaluation,  teams  should  meet  to  discuss  problems  they  found 
during  their  individual  investigation  using  the  stereoscopic  glasses  system.  This  supports 
the  first  empirical  finding,  that  design  teams  were  able  to  detect  more  errors  when  using 
the  stereoscopic  glasses  system.  This  will  allow  the  teams  to  assemble  in  a  more  team¬ 
like  environment  and  will  promote  interaction,  stimulation  of  new  ideas,  and  further 
detection  of  potential  design  problems.  It  is  suggested  that  during  this  phase  of  the  CDR 
process,  a  monitor  larger  than  21 -inch  be  used.  The  reason  for  the  larger  monitor  is  that 
when  more  than  two  or  three  people  are  viewing  the  stereoscopic  image  generated  from 
the  stereoscopic  glasses  and  emitter  system,  it  is  critical  that  viewers  are  in  a  frontal 
position  to  the  emitter  to  achieve  the  correct  3-D  perception.  However,  based  on  the 
subjective  data  collected  and  evaluated  in  this  study,  caution  must  be  taken  when  using 
the  stereoscopic  glasses  systems.  Users  did  not  perceive  the  value  of  this  technology  and 
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in  some  cases  experienced  eye  fatigue  when  using  it.  The  implication  of  using  the 
stereoscopic  glasses  systems  will  require  shorter  usage  times  and  they  should  not  be  used 
for  extended  CDRs.  It  is  also  suggested  that  better  navigational  methods  be  provided  to 
improve  the  likelihood  of  user  acceptance  of  this  technology  for  CDRs.  If  design  team 
members  were  able  to  move  quickly  to  a  selected  location,  the  time  in  the  system  could 
be  reduced  and  the  likelihood  of  user  acceptance  would  be  improved. 

Lastly,  after  exposure  to  the  conceptual  model  as  an  individual  and  as  a  team 
member,  teams  should  resolve  problems  with  their  improved  knowledge  base  using  face- 
to-face  communication.  At  this  phase  of  the  CDR  activity,  the  design  participants  have 
developed  an  improved  understanding  of  the  concept  model  and  the  potential  problems 
that  exist.  It  is  suggested  that  a  team  leader  facilitate  the  process  during  this  phase  of  the 
CDR.  This  leader  would  be  responsible  for  focusing  the  discussion,  keeping  team 
members  on  track,  and  helping  design  team  members  formulate  consensus  on  problem 
resolutions  in  real-time. 

Also,  when  using  virtual  visual  display  systems  for  CDR  activities,  it  is 
recommended  that  novice  users  get  adequate  exposure  to  the  technologies  prior  to 
conducting  the  actual  CDR.  This  will  eliminate  potential  problems  with  navigating  in  the 
virtual  environment  and  reduce  potential  anxiety  when  using  the  systems. 

The  problem  faced  in  creating  a  good  VE  is  important  to  the  success  of  the  virtual 
representation  of  a  concept  design.  The  problem  faced  in  creating  good  VEs  with  today’s 
tools,  is  that  the  translation  from  CAD  data  formats  into  virtual  world  software  formats  is 
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a  time  consuming  and  sometimes  difficult  process.  CAD  data  formats  contain  larger 
amounts  of  information  than  are  necessary  for  VE  developments.  User  comments 
obtained  in  this  study  indicate  that  some  users  expected  a  higher  level  of  detail  in  the 
virtual  model.  Yet  the  current  state  of  computer  technology  limits  the  level  of  detail  that 
is  feasible.  VE  models  require  less  detail  in  order  to  provide  the  correct  visual  rendering 
of  the  virtual  objects.  Less  detail  minimizes  latency,  problems  associated  with  slow 
processing  power.  Current  computer  processing  power  restricts  the  ability  to  have  a  one 
to  one  matching  between  CAD  and  VE  file  formats. 

Another  important  technology  issue  is  that  equipment  set-up  is  not  trivial. 

Required  are  computer  programming,  hardware  integration,  and  computer  trouble¬ 
shooting  skills.  This  issue  is  brought  up  because  it  is  important  to  realize  that  design 
teams  are  comprised  of  individuals  with  varying  levels  of  technical  expertise  who  most 
likely  do  not  have  all  the  skills  needed  to  develop  or  use  these  tools  without  the  assistance 
of  experienced  technical  specialists. 

In  addition,  it  is  strongly  recommended  that  future  design  tools  possess  the 
technical  characteristics  necessary  to  give  users  the  ability  to  move  and  interact  with  the 
data  model,  and  to  make  changes  dynamically  in  a  more  intuitive  manner.  User 
comments  indicate  that  a  future  tool  that  would  allow  them  to  dynamically  make  changes 
while  they  were  conducting  the  design  review  would  greatly  benefit  the  CDR  process. 
This  tool  could  be  a  VE-based  CAD  tool,  which  provides  users  with  the  capability  to 
make  design  changes  in  3-D  and  still  maintain  the  level  of  engineering  data  necessary  to 
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build  the  end  product.  An  extension  to  research  in  haptic  interface  applications  to  design, 
would  be  the  capability  to  dynamically  make  changes  to  the  virtual  design  by  grabbing  an 
object,  scaling  it,  moving  it,  and  making  final  decisions  during  the  CDR.  This  capability 
does  not  presently  exist  in  today’s  systems. 

User  Interface  Design  and  Usability  Engineering 

Understanding  the  task  and  who  will  be  using  the  technology  is  important  in 
selecting  which  display  system  is  appropriate.  It  is  critical  to  remember  that  not  all 
potential  users  have  experience  with  VEs  or  with  computer  technology,  and  new 
experiences  may  be  intimidating  at  first.  Shneiderman  (1993)  indicates  in  a  book  preface 
that  ‘lighting  for  the  user”  is  important  because  frustration  and  anxiety  are  a  normal  part 
of  daily  life  for  many  computer  users.  People  struggle  to  learn  command  language  and 
menu  selection  systems  that  are  supposed  to  help  them  do  their  job.  Some  people 
encounter  such  serious  cases  of  computer  shock,  terminal  terror,  or  network  neurosis  that 
they  totally  avoid  using  computers.  The  researcher  also  indicates  that  these  electronic- 
age  maladies  are  growing  more  common. 

Future  VE  tools  need  to  be  intuitive  to  the  user  and  need  to  closely  match  the  task 
they  are  trying  to  accomplish.  User  comments  obtained  in  this  study  identified  limitations 
of  current  computer-based  systems  and  offered  some  insight  into  possible  interface 
improvements.  Users  want  improved  resolution  of  displays,  improved  navigation 
techniques,  incorporation  of  touch  and  more  intuitive  ways  to  grasp  objects,  and  the 
inclusion  of  some  form  of  dynamic  modeling.  The  closer  systems  capabilities  match  the 
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real  world,  the  more  likely  VE  systems  will  give  users  the  better  performance 
capabilities.  Navigation  is  a  critical  function  in  VEs,  allowing  users  to  move  and  change 
points  of  view.  Current  systems  accomplish  this  task  by  using  methods  that  often  do  not 
closely  relate  to  natural  or  expected  actions  in  the  real  world.  Navigational  devices  must 
be  improved  if  they  are  to  become  more  widely  and  successfully  used.  This  researcher 
strongly  agrees  with  Bryson  (1996)  who  identified  two  primary  difficulties  impeding  the 
development  of  VR  applications.  They  axe:  VR  interfaces  are  a  new  paradigm  to  which 
2-D  interface  paradigms  do  not  easily  apply,  and  VE  systems  need  to  deliver  high 
performance  in  order  to  achieve  user  expectations. 

Recommendations  for  Future  Research 

On  the  basis  of  the  findings  and  observations  gathered  from  this  study,  recommendations 
for  future  research  in  two  major  areas  are  presented.  These  are:  applications  to  CDRs, 
and  user  interface  design  issues  and  usability  engineering  practices. 

Application  to  Concept  Design  Reviews 

This  study  examined  the  use  and  value  of  some  commercially  available  3-D  visual 
display  systems  to  the  CDR  process.  These  systems  will  continue  to  mature  and  evolve 
over  time. 

User  comments  obtained  in  this  study  have  also  lead  the  researcher  to  believe  that 
physical  interaction  with  a  virtual  model  is  necessary  during  CDR  activities.  Haptic 
devices  have  been  suggested  for  medical,  entertainment,  and  military  applications 
(Burdea,  1996)  because  they  provide  the  user  with  the  sense  of  touch.  These  haptic 
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devices  may  also  have  the  potential  to  provide  interesting  insight  into  whether  design 
team  members  would  gain  improved  performance.  With  this  capability  design  team 
members  may  intuitively  grasp,  touch,  and  move  objects  -  much  more  than  can  be 
accomplished  using  current  navigational  techniques  such  as  a  3-D  mouse. 

Burdea  (1996)  also  found  that  the  sense  of  presence  might  serve  an  important  role 
in  VE-based  design  review  activities.  Researchers  have  initiated  efforts  to  develop  an 
understanding  of  the  sense  of  presence  in  virtual  environments  but  much  of  this  work  has 
not  yet  been  focused  on  specific  application  areas.  A  better  understanding  of  how 
important  the  sense  of  presence  is  in  product  design  review  environments  would  be 
beneficial. 

The  inclusion  of  more  technologies  (e.  g.,  CAVE,  ImmersaDesk,  Immersive 
WorkBench,  and  holographic  imaging  systems)  for  testing  is  needed  as  these 
technologies  mature  and  become  more  affordable  for  CDR  use.  As  these  tools  enter  the 
commercial  market  it  is  necessary  for  organizations  to  understand  the  performances  and 
task  utilization  of  these  systems  in  order  to  make  correct  trade-off  determinations. 

Lastly,  this  study  was  based  on  the  performance  of  U.  S.  Army  design  teams.  Test 
replication  in  other  environments  (e.g.,  NASA  space  element  design,  Air  Force  or  Navy 
weapon  system  development  review,  and  civilian  consumer  product  development)  is 
needed  to  provide  validation  and  understanding  for  the  entire  product  design  community. 
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User  Interface  Design  and  Usability  Engineering 

“Knowing  the  user”  is  one  of  the  most  fundamental  of  all  human  interface 
usability  guidelines  (Shneiderman,  1993;  Nielsen,  1993).  Future  studies  focusing  on  the 
“people”  in  the  product  development  community  are  needed  to  gain  a  better 
understanding  of  the  differences  between  product  design  review  participants,  the 
dynamics  of  team  interaction,  the  complexity  of  the  process,  and  the  required  tasks. 

These  discoveries  could  then  be  translated  into  improved  VE  interface  designs  for  the 
general  product  design  and  development  community.  Nielsen  (1993)  suggests  that  more 
formal,  systematic  task  and  functional  analyses  can  be  fruitful,  providing  interface 
designers  with  information  regarding  what  users  want  to  accomplish,  how  they  will 
accomplish  it,  and  what  the  relationships  are  between  the  tasks  and  the  actions  in  the 
application.  These  methods  can  help  determine  the  correct  functionality  of  the  interface 
and  help  identify  occasional  and  exceptional  tasks  which  are  more  difficult  to  discover 
(Shneiderman,  1993). 

One  issue  that  was  not  addressed  in  this  study  is  the  effect  of  differing 
personalities  among  design  team  participants.  These  personality  differences  may  have  an 
effect  on  how  users  performed  and  how  they  felt  about  the  visual  display  systems.  An 
approach  that  merits  further  investigation,  as  described  by  Shneiderman  (1993),  is  to 
focus  on  measuring  the  cognitive  and  personality  styles  of  the  product  design  team 
community  and  translating  these  findings  into  improved  VE  interface  designs. 
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Summary 


Demands  and  technological  advances  for  sophisticated  methods  of  collaborative 
interaction  have  led  to  great  interest  in  the  techniques  and  applications  of  3-D  imagery. 
Few  researchers  and  organizations  have  the  opportunity  to  make  direct  comparisons  of 
different  3-D  systems  used  for  CDRs.  Because  of  this  limitation,  their  choice  of  display 
systems  maybe  arbitrary.  This  study  was  conducted  to  evaluate,  through  comparative 
empirical  testing  and  data  analysis,  how  commercially-available  virtual  3-D  display 
systems  contribute  to  cross-functional  team  collaboration  in  a  product  design  review 
environment. 

Applications  of  VR  to  the  product  design  and  development  process  have  intrigued 
many.  A  majority  of  today’s  organizations  are  exploring  the  possibility  of  designing 
products  only  in  the  virtual  sense.  VE  technology  allows  for  persons  to  experience  a  new 
product  concept  prior  to  building  costly  physical  prototypes.  Through  the  use  and  testing 
of  four  different  visual  display  systems,  the  results  in  this  study  serve  to  strengthen  the 
premise  that  VEs  improve  the  effectiveness  and  efficiency  of  the  design  review  process. 
Data  collection,  which  applied  the  four  visual  display  systems  in  an  experimental  setting, 
produced  results  identifying  the  strengths  and  weaknesses  of  each  visual  display  system. 
These  findings  are  important  because  they  lead  to  a  better  understanding  of  the  trade-off 
required  when  selecting  a  VE  visual  display  system,  and  they  provide  insight  for 
consideration  and  testing  of  future  interface  designs. 
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Abbreviation  of  Terms 
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Abbreviation  of  Terms 


BOOM 

Binocular  Omni-Orientation  Monitor 

CAD 

Computer  aided  design 

CAE 

Computer-aided  engineering 

CAM 

Computer-aided  manufacturing 

CAPP 

Computer-aided  process  planning 

CAVE 

Cave  Automatic  Virtual  Environment 

CDR 

Concept  Design  Review 

CRT 

Cathode  Ray  Tube 

CSCW 

Computer  Supported  Cooperative  Work 

DC 

Direct  Current 

DMD 

Digital  Micromirror  Device 

DOD 

Department  of  Defense 

FOV 

Field  of  View 

HDTV 

High-definition  television 

HMD 

Head  Mounted  Display 

IR 

Infrared 

LAN 

Local  Area  Network 

LCD 

Liquid  Crystal  Display 

NASA 

National  Aeronautics  Space  Administration 

PC 

Phase  Coherent 

QUIS 

Questionnaire  for  User  Interface  Satisfaction 

SE 

Synthetic  Environment 

SGI 

Silicon  Graphics  Incorporated 

TACOM 

Tank-automotive  and  Armaments  Command 

TOF 

Time  of  Flight 

TFTS 

Tracked  Fuel  Trailer  System 

VE 

Virtual  Environment 

VP 

Virtual  Prototyping 

VR 

Virtual  Reality 

WAN 

Wide  Area  Network 

2-D 

Two-dimensional 

3-D 

Three-dimensional 
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Tracked  Fuel  Trailer  System  (TFTS)  Requirements 
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TRACKED  FUEL  TRAILER  SYSTEM  (TFTS) 


Concept  Philosophy 

A  new  tactic  of  fighting  being  evaluated  is  to  engage  two  opposing  forces  within  a  range 
of  50  to  100  kilometers.  The  way  to  defeat  an  Army  using  this  strategy  would  be  to 
disrupt  the  logistics  resupply  line.  DSOPS  believes  that  digital  voice  and  data 
communications  would  enable  this  to  work.  This  requires  the  ability  for  ground  vehicle 
systems  to  continually  be  on  the  move.  Providing  extended  range  capability  and 
minimizing  logistic  supply  interruptions. 


REQUIREMENTS 

•  600  Gallon  Fuel  Pod 

•  Trailer  to  be  towed  behind  a  tracked  (Abrams  Ml  Main  Battle  Tank)  or 
tactical  wheeled  vehicle 

•  Continuously  provide  fuel  to  the  prime  mover  when  in  tow 

•  Remotely  activated  &  quick  disconnect  of  the  fuel  delivery  system  and  trailer 

•  Convertible  for  water  and/or  palletized  load 

•  Capable  of  external  helicopter  lift 

•  Towed  safely  at  speeds  of  prime  mover 

•  Equipped  with  essential  operational  safety  features 

•  Recoverable  and  reusable  without  repair  or  preparation 
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Test  Plan  Summary 
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Test  Plan  Summary 


1 .  Participants: 

1.1.  Participants  for  the  experimentation  are  personnel  at  the  U.  S.  Army  Tank- 
automotive  and  Armaments  Command  in  Warren,  Michigan.  Specifically, 
the  following  operational  organizations;  New  Equipment  Training,  Human 
Factors,  Safety,  Logistics,  Maintenance,  Quality  Assurance,  and  Design 

1.2.  Test  participants  will  fill  out  VE  survey  one  (1),  Participant  Background 
Data  prior  to  any  experimentation.  The  survey  contains  data  on 
educational,  professional,  and  computer  literacy  of  each  participant. 

1 .3.  Participants  will  be  tested  for  color  deficiency  and  visual  acuity  prior  to 
experimentation  using  a  standard  Ishihara  Color  Deficiency  test  and  a 
Snelling  Chart  examination.  Tests  take  5-10  minutes  each. 

1 .4.  Subjects  that  are  tested  positive  for  color  deficiency  will  be  eliminated 
from  the  subject  pool. 

2.  Design  Team  Composition: 

2. 1 .  Identification  of  representative  design  team  members.  Teams  are 
composed  of  designers  and  non-designers.  A  total  of  3  subjects  for  each 
team;  1  designer  and  2  non-designers. 

2.2.  Test  participants  will  be  pre-assigned  to  a  design  team,  based  on  data  from 
the  VE  Assessment  Survey  One.  The  purpose  is  to  create  teams  with 
similar  backgrounds  &  characteristics  based  on  an  aggregation  of 
experience,  computer  literacy,  and  education.  Balancing  the  teams. 

2.3.  The  designer  will  act  as  the  team  leader. 

3.  Pilot  Test: 

3.1.  Conduct  orientation  and  pilot  test  in  one  ( 1 )  day. 

3.2.  The  pilot  test  will  be  done  at  on-site  laboratory.  The  purpose  of  the  pilot 
test  is  to  evaluate  test  procedures,  VE  assessment  surveys,  and  test  model. 

3.3.  The  test  will  use  the  same  procedures  and  data  collection  mechanisms  as 
the  experimentation.  Based  on  the  pilot,  modifications  to  the  procedures 
and  surveys  will  be  done. 

4.  Experimental  Design: 

4. 1 .  4X4  Graeco-Latin  Square 

4.2.  Four  variables  each  with  four  levels.  Teams  =  Ti,  Ti,  T3,  T^;  Experiment 
Order  =  E|,  Ei,  E3,  E»;  Technology  =  A,  B,  C,  D;  and  Subassembly  =1.2. 
3,4 

4.3.  Design  teams  will  be  randomly  assigned  to  a  test  sequence,  a  column  in 
the  Graeco-Latin  Square. 

4.4.  Time  order  will  be  randomly  assigned  to  a  row  in  the  Graeco-Latin  Square 

4.5.  Where  Technology  A,  B,  C,  D  are  defined  as  HMD,  BOOM,  Stereoscopic 
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4.6. 


Glasses,  and  Monoscopic  3-D  respectively. 

Where  Subassemblies  1,  2,  3,  4  are  defined  as  fuel  transfer  subsystem,  fuel 
container  subsystem,  towing  subsystem,  and  suspension  subsystem 
respectively. 

4.7.  Purposeful  assignment  to  prevent  confounding. 

5.  Subassemblies  (problem/issues): 

5. 1 .  Four  (4)  different  subassemblies  of  the  concept  trailer  will  be  used  for  the 
experimentation.  Each  subassembly  contains  problems  or  issues  with 
similar  levels  of  complexity. 

5.2.  The  subassembly  issues  are  related  to  mechanical,  ergonomic, 
maintenance,  or  operational  issues. 

6.  Experiment 

6. 1 .  Pre-test  Orientation: 

6.1.1.  Conduct  pre-test  orientation  on  the  day  of  the  experiment  in  a  local 
conference  room. 

6.1.2.  Orientation  Agenda: 

6. 1.2.1.  General  purpose/objective  of  the  experiment  (i.e. 
the  purpose  is  evaluate  the  usefulness  of  a  variety  of  VE 
based  visualization  technologies  on  the  product  design 
review  process) 

6. 1 .2.2.  Describe  Tracked  Fuel  Trailer  System  requirements 

6. 1.2.3.  Provide  an  overall  schedule  of  day’s  events 
(logistics) 

6. 1 .2.4.  Description  of  Test  Conduct  (Protocol) 

6.2.  Test  Protocol: 

6.2. 1 .  Groups  will  move  between  technology  stations. 

6.2.2.  Groups  will  have  a  time  limit  (one-hour)  at  each  technology 
station. 

6.2.3.  Each  technology  station  will  have  a  technology  test  observer. 

6.2.4.  System  familiarization  (training)  will  be  conducted  at  each 
technology  station 

6.2.4. 1.  Standard  15  minute  training  session 

6.2.4.2.  An  alternate  model  will  be  used  for  training. 

6.2.5.  Design  reviews  will  be  conducted  at  each  technology  station 

6.2.6.  Test  Teams  will  be  told  subsystem  of  focus  at  each  technology 
station. 

6.2.7.  Test  Teams  will  be  told  that  there  are  at  least  3  problems  or  issues 
in  each  subassembly. 

6.2.8.  The  designer  will  act  as  the  team  leader  and  discussion  facilitator. 

6.2.9.  Participants  will  be  asked  to  identify  an  issue,  and  to  resolve  the 
issue  before  moving  on  to  the  next  problem  (sequential).  The 
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purpose  is  to  reach  consensus  on  problem  identification  and 
possible  resolution 

6.2. 10.  It  is  suggested  that  each  team  use  the  technology  in  anyway  they 
feel  is  appropriate. 

6.2. 1 1 .  The  test  observer  will  take  continuous  clock  times  of  problem 
identification  times  and  resolution  times.  Documented  on  a 
standard  quantitative  data  Collection  Sheet. 

6.2.12.  The  test  observer  will  document  solutions. 

6.2.13.  The  test  observer  will  document  anything  unusual  that  may  effect 
the  data 

6.2.14.  After  experimentation,  each  person  will  complete  VE  Assessment 
Survey  Two  —  Individual  Technology.  Participants  will  be  given 

1 5  minutes  to  complete  the  survey 

6.2.15.  After  completion  of  all  technologies,  each  participant  will 
complete  VE  Assessment  Survey  Three  --  Technology 
Comparisons.  Participants  will  be  given  15  minutes  to  complete 
the  survey 
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Appendix  D 


VE  Assessment  Survey  One  —  Participant  Background  Data 
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Virtual  Environment  Assessment  Survey  One 

Participant  Background  Data 


Name: _ 

Participant  Number _ 

Part  1.1  Personal  Information 


Sex:  _ Male  _ Female 

1.1.1  Do  you  wear  corrective  glasses  (not  contact  lenses)? _ Yes  _ No 

1.1.2  Have  you  ever  experienced  simulator  (motion)  sickness? _ Yes  _ No 

Part  1.2  Educational  Background 


1 .2. 1  Identify  the  highest  level  of  your  education 

_  High  School  _ Bachelors  Degree  _  Doctoral  Degree 

_  Associates  Degree  _  Masters  Degree  _  Post  Graduate  Study 

1 .2.2  Identify  the  field  of  your  study 

_ Engineering  _ Business  _ Quality  Assurance  _ Psychology 

_  Other  What  is  your  field  of  study?  _ 

Part  1.3  Professional  Experience 


1.3.1  How  many  years  have  you  been  working  in  your  field? 

_  Less  than  one  year  _  3  to  5  years  _  10-15  years 

_ 1  to  3  years  _  5-10  years  _  Greater  than  15  years 

1.3.2  In  what  area  have  you  spent  most  of  your  career? 

_  Project  Management  _  Research  _ Engineering 

_  Manufacturing  _ Quality 

_  Other  What  area?  _ 

Part  1.4  Computer  Usage 


1.4.1  How  many  different  types  of  computer  systems  (e.g.  main  frames  and  personal 
computers)  have  you  worked  with? 

_ None  _ 1  _ 2  _ 3-4  _ 5-6  _ more  than  6 
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1 .4.2  Of  the  following  devices,  software,  and  systems,  check  those  that  you  have 
personally  used  and  are  familiar  with: 


. keyboard 
.  numeric  key  pad 
.  track  ball 
.  mouse 
.  light  pen 
.joy  stick 
touch  screen 


_ text  editor 

_ word  processor 

_ file  manager 

_ electronic  spreadsheet 

_ electronic  mail 

_ graphics  software 

_ computer  games 


_ color  monitor 

_ time-share  system 

_ workstation 

_ personal  computer 

_ floppy  drive 

_ hard  drive 

_ compact  disk  drive 


1.4.3  Of  the  following  types  of  visualization  devices  and  systems,  check  those  that  you 
have  personally  used  and  are  familiar  with: 

_ Helmet  Mounted  Display  (HMD)  _  Stereoscopic  Glasses 

_ Binocular  Omni-Orientation  Monitor  (BOOM)  _  3-D  Solid  CAD  Models 


Part  1.5  Design  Review  Experience 


1.5.1  How  many  design  and/or  concept  reviews  have  you  participated  in? 
_ None  _ 1  _ 2  _ 3-4  _ 5-6  more  than  6 


Please  leave  blank,  a  standard  eye  examination  and  color  deficiency  test  will  be  given 
prior  to  the  experimentation. 

Visual  Acuity _ 

Color  Deficiency  _ 
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Appendix  E 


Test  Presentation 
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Appendix  F 


Outside  Observer  Comments 


Technology  A.  HMD 


User  wants  to  point  at  a  certain  location  in  the  design,  and  at  first  doesn’t  think  that  it  is 
possible  then  notices  the  virtual  hand  makes  pointing  easy. 

User  commented  that  he  was  just  as  comfortable  viewing  the  computer  monitor  and 
didn’t  want  to  use  the  HMD. 

Participants  had  used  the  HMD  and  pointing  devices  fairly  easily  to  locate  problem  areas 
and  to  bring  others  into  individual  perspectives. 

Participants  followed  design  protocol,  but  it  was  difficult  for  them  to  focus  on  one  issue, 
sometimes  they  sidetracked  into  several  issues,  but  where  brought  back  into  focus 
by  the  team  leader. 

Issues  that  were  solved  were  sometimes  re-opened 

Team  made  a  summary  at  the  end  of  the  session,  recapping  their  conclusions. 

The  person  in  the  HMD  seemed  isolated  from  the  conversation.  The  two  persons  looking 
at  the  monitor  where  engaged  in  conversation  and  the  HMD  user  was  busy 
looking  around  the  model.  HMD  User  seemed  intrigued  with  the  technology  and 
was  not  directing  the  others  to  specific  problem  areas.  HMD  user  would  interact 
once  in  awhile. 

User  used  physical  hand  gestures  to  explain  his  idea  to  his  partners.  He  never  used  the 
virtual  hand  to  point  and  explain  his  perspective. 

Users  sometimes  re-iterated  their  findings  to  the  HMD  user  after  exchanging  positions. 

Users  asked  the  HMD  user  to  navigate  to  a  certain  part  of  the  model. 

When  switching  the  HMD  from  design  member  some  users  lost  their  place  in  the  model. 
Not  enough  experience  with  how  these  systems  work  and  didn’t  realize  he  was 
out  of  the  trackers  range.  Simply  fixed  when  he  moved  forward. 

Some  users  didn’t  navigate  around  in  the  model;  they  remained  in  a  fixed  position.  I  think 
he  didn’t  want  to  disrupt  what  the  remaining  members  were  looking  at  and 
discussing. 

Teams  kept  conversing  without  using  the  HMD.  This  seems  typical  for  design  reviews, 
people  look  at  a  concept  then  prefer  to  discuss  with  face  to  face  contact.  In  fact 
the  model  wasn’t  even  in  focus  on  the  external  monitor. 

User  made  a  comment  that  all  the  team  members  should  have  HMDs. 

HMD  user  asked  the  others  to  navigate  him  to  where  they  wanted  to  look. 

Group  outside  of  the  HMD  seemed  frustrated  because  the  person  in  the  HMD  was  busy 
looking  at  what  he  wanted  and  kept  moving  around  especially  when  they  were 
trying  to  point  to  and  discuss  a  specific  thing. 

One  team  member  made  a  comment  that  this  is  definitely  not  good  for  group  interaction, 
or  group  work.  “The  guy  in  the  HMD  is  like  in  the  closet” 

The  team  member  in  the  HMD,  flipped  up  his  visor  so  he  could  be  part  of  the  discussion, 
then  the  dialogue  continued  with  gestures  and  face  to  face  discussions. 

Most  of  the  discussion  was  without  using  the  HMD 

Users  want  to  focus  on  a  detailed  level  design  and  not  a  conceptual  level. 
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Technology  B,  BOOM 


Users  had  increased  difficulty  in  maneuvering  in  the  virtual  design  space.  Especially 
when  it  comes  to  leveling  your  personal  height  in  the  world.  Sometimes  they 
couldn’t  get  the  exact  view  they  wanted  and  discussed  issues  from  their  best 
attempt 

Comments  on  difficulty  to  use 

Comments  on  grainy  resolution 

One  user  commented  that  it  is  easier  to  move  around  by  looking  at  the  monitor  and 

physically  moving  the  boom  device.  He  preferred  to  focus  on  the  monitor  and  use 
the  boom  as  the  navigation  device. 

Teams  seem  to  be  quite  comfortable  talking  around  the  monitor,  without  any  stereoscopic 
perspective. 

After  a  period  of  time  the  users  did  not  use  the  system  as  the  focus  of  their  discussion. 
They  kept  the  model  in  view  of  the  monitor,  by  one  person  holding  the  device 
steady.  Discussions  continued  without  the  BOOM  device,  allowing  more  face  to 
face  interaction. 

Not  every  one  used  the  technology  for  a  long  duration  of  time,  they  preferred  to  look  at 
the  monitor. 

One  user  commented  that  the  BOOM  seemed  more  intuitive  to  operate  and  navigate. 

One  user  commented  that  the  BOOM  user  the  odd  man  out,  this  is  a  problem  with  this 
technology. 


Technology  C,  Stereoscopic  Glasses 


Participants  made  comments  on  the  quality  of  the  visual  system. 

Using  single  monitor  participants  seemed  very  comfortable  and  the  dialogue  flowed. 
There  was  no  need  for  participants  to  switch  devices,  or  places,  or  seats. 

Some  users  took  off  their  glasses  towards  the  end  of  the  session,  they  continued  to 

discuss  problems  and  issues.  They  would  put  the  glasses  when  they  want  to  point 
at  something. 

All  participants  were  able  to  view  the  same  monitor.  It  seemed  easier  for  them  to  share, 
or  take  charge  of  the  discussion,  or  show  an  individual  perspective.  This  differed 
from  the  HMD  where  the  HMD  user  was  looking  at  what  he  wanted  to 
independent  of  the  group  discussion. 

In  one  case,  one  user  pointed  at  a  specific  part  of  the  model  without  using  his  glasses. 

Without  the  glasses  the  monitor  picture  appears  distorted  and  offset,  but  he  didn't 
put  the  glasses  back  on  to  get  his  point  across. 

Participants  had  difficulty  navigating  with  the  mouse  device. 

One  participant  commented,  “does  anyone  know  how  to  move  around?"... ”3-D  might  be 
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good  but  it  seems  you  can’t  see  the  detail,  maybe  the  dimension  is  more  a 
problem  than  its  worth.” 

The  pintle  seems  very  detailed  and  the  glasses  make  it  hard  for  them  to  actually  see  the 
detail,  eyestrain. 

Participants  used  gestures  and  natural  pointing  when  discussing  problems  and  solutions. 

One  participant  commented  in  relation  to  3-D  drawings... ’’that  if  he  had  a  3-D  drawing 
he  would  know  immediately  how  the  pintle  worked.” 

One  participant  said  he  would  prefer  to  have  the  monitor  divided  with  the  same  views  as 
3-D  drawings. 


Technology  D,  Monoscopic  CRT  Monitor 


All  design  team  members  seemed  comfortable 

Sometimes  participants  continued  discussing  issues  without  even  looking  at  the  monitor, 
they  preferred  to  make  eye  contact  with  each  other. 

Using  a  single  monitor  participants  seemed  very  comfortable  and  the  dialogue  followed. 
There  was  no  need  for  participants  to  switch  devices,  or  places,  or  seats. 

Participants  frequently  used  physical  hand  gestures  to  describe  their  issue  or  problem. 
Often  pointing  to  the  monitor  to  specifically  point  out  what  they  wanted  to 
discuss. 

Difficulty  using  mouse  device. 

Discussions  where  natural  with  face  to  face  dialogue  and  gesturing.  You  get  the  sense 
that  the  participants  felt  as  if  this  is  more  appropriate. 

Participants  used  pointing  and  natural  gestures  to  help  explain  their  position. 

A  lot  of  face  to  face  dialogue 

When  looking  at  the  regular  monitor,  it  appears  that  the  participants  love  to  discuss  all 
kinds  of  multi-issues 

After  a  period  of  looking  at  the  model,  the  discussions  continued  without  looking  at  the 
model  itself.  Tne  group  was  in  deep  conversation  about  the  issues  without  the 
model  assisting  them. 
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Appendix  G 


Data  Collection  Sheet 
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Quantitative  Data  Collection  Sheet 


This  form  will  be  used  to  collect  data  during  group  experimentation  with  each  of  the 
technologies.  This  information  will  be  collected  by  trained  observers. 

Identification  number:  Group _ Technology _  Subassembly _ 

Date _ 

Start  Time  _ 


Problem/Issues 

Resolution 

Detection  Time 
(mins) 

Resolution 
Time  (mins) 

Did  the  design  team  follow  the  suggested  test  protocol?  _ Yes  _ No 

If  no,  what  strategy  did  they  use? 


How  did  the  design  team  use  the  technology?  (Did  each  person  use  the  technology 
before  consensus?) 
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Provide  comments  regarding  the  experimental  sessions. 
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Appendix  H 


VE  Assessment  Survey  Two  --The  Individual  Technology 
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Virtual  Environment  Assessment  Survey  Two 

Individual  VE  Technology 


This  survey  will  be  administered  after  completion  of  each  individual  technology 
concept  design  review. 

Identification  number:  Group _  Participant _  Subassembly _ 

Visualization  Technology: 

_ HMD  _ Stereoscopic  Glasses 

_ BOOM  _ 3-D  Monoscopic  CAD 

For  the  following  questions  please  circle  the  number  which  most  appropriately 
reflect  your  impressions  about  this  visualization  technology.  NA  =  Not  Applicable 

Part  2.1  Training  


2.1.1  Was  the  training  you  received  on  the 
visualization  technology  adequate? 

Insufficient 

1  2  3 

4 

Sufficient 

5  NA 

2.1.2  Did  you  understand  the  correct 
operating  procedures  for  the  technology? 

Misunderstood 

1  2  3 

4 

Fully  understood 

5  NA 

2.1.3  Were  you  sufficiently  prepared  to 
conduct  a  concept  review  exercise? 

Insufficiently  prepared 

1  2  3 

4 

Totally  prepared 

5  NA 

2.1.4  Learning  to  operate  the  system  was: 

difficult 

l  2  3 

4 

easy 

5  NA 

Provide  comments  regarding  the  training  you  received  prior  to  the  test  phase. 


Part  2.2  The  Concept  Design  Review  Process 


2.2. 1  Did  the  technology  assist  you  in 
conducting  your  team  concept  review? 

Not  at  all 

1  2 

3 

Totally  assisted 

4  5  NA 

2.2.2  Did  the  technology  assist  in  detecting 

Not  at  ail 

Totally  assisted 

design  problems  or  issues? 

1 

2 

3 

4  5  NA 

2.2.3  Did  the  technology  help  in  describing 
issues  to  your  team  members 

Not  any  help 

1  2 

3 

Totally  helpful 

4  5  NA 

2.2.4  Did  the  technology  stimulate 

Not  at  all 

Totally  stimulating 

creativity  and  problem  solving? 

1 

2 

3 

4  5  NA~ 
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2.2.5  Did  the  technology  assist  group 
interaction  and  discussion? 

Not  at  all  Totally  Assisted 

1  2  3  4  5  NA 

2.2.6  Did  the  technology  help  develop  a 
group  consensus? 

Not  at  all  Totally  helped 

1  2  3  4  5  '  NA 

2.2.7  Did  the  technology  provide  a  better 
understanding  of  the  concept 
(configuration)? 

Not  at  all  Totally  Assisted 

1  2  3  4  5'  NA 

2.2.8  Did  the  technology  provide  easier 
recognition  of  form  &  pattern? 

Not  at  all  Totally  Easier 

1  2  3  4  5  NA 

2.2.9  If  this  technology  became  a  standard 
for  concept  design  reviews  do  you  think 
that  the  product  development  time  would 
decrease? 

Not  at  all  Decreased  time 

1  2  3  4  5  NA 

2.2.10  If  this  technology  became  a  standard 
for  concept  design  reviews  do  you  think 
that  the  final  product  quality  would  be 
improved? 

Not  at  all  Totally  improve  quality 

1  2  3  4  5  NA 

Provide  comments  regarding  the  effect  of  this  visualization  tool  to  perform  concept 
design  review  tasks. 


Part  2.3  Quality 


2.3.1  Was  the  technology  helpful  as  a 
common  platform  for  group  discussion? 

Not  at  all  Totally  helpful 

1  2  3  4  5  NA 

2.3.2  How  useful  do  you  think  this 
technology  is  in  the  product  design  review 
process? 

Not  at  all  Totally  useful 

1  2  3  4  5  NA 

2.3.3  Did  you  feel  that  your  time  using  this 
technology  was  well  spent? 

Waste  of  time  Totally  well  spent 

1  2  3  4  5  NA 

2.3.4  Did  you  feel  that  you  had  adequate 
time  using  this  technology? 

Inadequate  Adequate 

1  2  3  4  5  NA 

2.3.5  Did  you  feel  that  any  dme  was 
wasted  waiting  for  use  of  the  technology? 

Wasted  time  No  time  wasted 

1  2  3  4  5  NA 
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2.3.6  Does  this  technology  positively 
affect  the  quality  of  the  design  review 
process? 

Negative  affect 

1  2 

3 

4 

Posiuve  affect 

5  NA 

2.3.7  Overall  reactions  to  the  system: 

Terrible 

I  2 

3 

4 

Wonderful 

5  NA 

Frustrating 

1  2 

3 

4 

Satisfying 

5  NA 

Dull 

1  2 

3 

4 

Stimulaung 

5  NA 

2.3.8  Operating  the  system  was: 

Difficult 

1  2 

3 

4 

Easy 

5  NA 

2.3.9  How  many  hours  a  day  are  you  willing  to  use  this  technology? 

_ none  _ 2  _ 4  _ 6  _ 8 

_  1  _ 3  _ 5  _ 7  _ more  than  8 


2.3. 10  Do  you  believe  this  technology  is  more  beneficial  for  individual  use? 

_ Yes  _ No 

If  yes.  Why? 


Provide  comments  regarding  the  quality  of  the  technology  as  it  applies  to  product  design 
and  your  overall  reaction  to  the  technology. 


Part  2.4  Physiological 


2.4. 1  Is  the  system  mentally  demanding? 


undemanding 
1  2 


3  4 


demanding 
5  NA 


2.4.2  Do  you  have  any  of  the  following  symptoms  (Check  all  applicable)? 


_ General  discomfort 

_ Fatigue 

_ Headache 

_ Eye  strain 


_ Fullness  of  head 

_ Blurred  vision 

_ Dizzy  (eyes  open) 

_ Dizzy  (eyes  closed) 


_ Vertigo 

_ Stomach  awareness 

_ Burping 

Nausea 
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Difficulty  focusing 
Difficulty  concentrating 


Increased  salivation 


Sweating 


Provide  comments  regarding  your  physical  or  emotional  state  after  using  this  technology. 


Part  2.5 


Is  there  any  aspect  of  the  visualization  tool  that  could  be  improved  to  enhance  the 
concept  design  review  activity?  _ No  _ Yes  If  yes,  please  explain 
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Appendix  I 


VE  Assessment  Survey  Three  —  Technology  Comparison 
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Virtual  Environment  Assessment  Survey  Three 

VE  Technology  Comparison 

This  survey  will  be  administered  after  completion  of  aU  individual  technology 
concept  design  reviews  have  been  completed. 

Identification  number:  Group _  Participant _ 

Part  3.1  Comparison 


3.1.1  Please  rank  the  usefulness  of  the  four  visualization  technologies  on  the  concept 
design  review  and  evaluation  process,  where  one  (1)  is  considered  the  most  useful 
and  four  (4)  is  the  least  useful. 

_ HMD  _ BOOM  _ Stereoscopic  Glasses  _ 3-D  Monoscopic  CAD 

Model 

3.1.2  Please  rank  the  difficulty  of  using  the  technologies  on  the  task  of  concept  design 
review  and  evaluation,  where  one  (1)  is  considered  the  most  difficult  and  four  (4)  is 
the  least  difficult. 

_ HMD  _ BOOM  _ Stereoscopic  Glasses  _ 3-D  Monoscopic  CAD 

Model 

3.1.3  Please  rank  the  practicality  of  using  the  technologies  for  concept  design  and 
evaluation,  where  one  (1)  is  considered  the  most  practical  and  four  (4)  is  the  least 
practical. 

_ HMD  _ BOOM  _ Stereoscopic  Glasses  _ 3-D  Monoscopic  CAD 

Model 

3.1.4  Please  rank  the  stimulation  of  group  interactivity  when  using  the  technologies  for 
concept  design  review  and  evaluation,  where  one  (l)  is  considered  the  most 
stimulating  group  interaction  and  four  (4)  is  the  least  stimulating  group  interaction. 

_ HMD  _ BOOM  _ Stereoscopic  Glasses  _ 3-D  Monoscopic  CAD 

Model 

3.1.5  Please  rank  the  usefulness  for  developing  group  consensus  when  using  the 
technologies  in  concept  design  review  and  evaluation,  where  one  (l)  is  considered 
the  most  useful  for  developing  group  consensus  and  four  (4)  is  the  least  useful  for 
developing  group  consensus. 

_ HMD  _ BOOM  _ Stereoscopic  Glasses  _ 3-D  Monoscopic  CAD 

Model 
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Part  3.2 


Answer  the  following  question:  In  retrospect,  after  experiencing  all  four  visualization 
technologies  would  you  make  any  changes  to  your  previous  responses  to  your  individual 
technology  surveys?  If  yes,  explain. 
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Data  for  Design  Problem  Detection  Time  and  Resolution  Time 
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Team 

Experiment  Order 

Technology 

Subassembly 

T, 

E, 

A 

i  ! 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

The  angle  of  the 
transfer  boom  has  too 
much  play  in  it. 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

30 

118 

30 

88 

How  is  the  fuel 
pumped? 

Need  mechanical  fuel 
pump  inside  of  or  in  the 
bottom  of  the  fuel  pod? 

130 

300 

12 

170 

The  angle  of  the 
transfer  boom  has  too 
much  play  in  it. 

Use  a  flexible  hose  that 
will  allow  quick  connect 

318 

353 

18 

35 

The  exit  point  of  the 
fuel  pod  is  too  high, 
may  cause  improper 
fuel  release. 

Move  the  exit  point  to 
the  bottom  or  front  side 
of  the  fuel  container 

464 

473 

III 

9 

Team 

Experiment  Order 

Technology 

Subassembly 

T, 

Ei 

B~ 

3 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

There  will  be  a 
problem  when  turning 
the  Ml  with  the  trailer 
attached  (problem  with 
the  turning  radius. 

Extend  the  pintle  on  the 
Ml  tank 

120 

150 

120 

30 

There  is  no  quick 
release  mechanism  on 
the  hitch  itself. 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

210 

420 

60 

210 

What  kind  of  power  is 
going  to  the  trailer,  no 
24V  cable 

Provide  the  trailer  with 
an  electrical  harness  that 
will  provide  24V 

450 

477 

30 

27 

The  lift  handles  do  not 
look  adequate  to  carry 
the  entire  weight  of  the 
system 

Need  to  strengthen  the 
handles  using  a  different 
lift  handle  design  in 
order  to  allow  for  air 
transportability 

599 

921 

122 

322 

230 


There  is  no  quick 

Provide  a  quick  release 

960 

1242 

39 

282 

release  mechanism  on 

in  the  electncal  harness 

the  hitch  itself. 

of  the  trailer 

Team 

Experiment  Order 

Technology 

Subassembly 

T, 

H3 

c 

4 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

It  appears  that  there 
will  be  torsion  on  the 
axle  which  can  cause 
excessive  twisting  on 
the  leaf  springs 

Utilize  the  same  axle 

195 

655 

195 

460 

Should  the  beams 
between  the  wheels  be 
changed? 

Use  a  fixed  beam  system 

664 

780 

9 

1 16 

Why  re-invent  the 
suspension  for  the 
trailer  system? 

Use  an  existing  trailer 
system's  suspension 

822 

860 

42 

38 

Is  the  leaf  spring 
system  adequate  to 
carry  the  load? 

Incorporate  a  shock 
absorber  system 

930 

979 

70 

49 

What  type  of  brake 
system  is  on  the 
trailer? 

Need  some  method  of 
braking  the  trailer 
system,  especially  if 
being  pulled  behind  a 
wheeled  tactical  vehicle. 

998 

1088 

19 

90 

Need  to  drop  on  the 
move  for  survival 
purposes. 

Need  some  type  of  skid 
plate  to  allow  you  to 
drop  and  continuously 
move 

1413 

1453 

325 

40 

At  what  speed  is  it 
safe  to  drop  the  trailer 
system 

Need  to  look  at  more 
detailed  operational 
requirements  for  the 
move  dropping  speeds 

1467 

1610 

14 

143 

231 


Team 

Experiment  Order 

Technology 

Subassembly 

T, 

E4 

D 

2 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

The  profile  of  the  tank 
is  too  high 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

30 

38 

30 

8 

Need  a  method  to  fill, 
drain  and  vent  the  fuel 

Incorporate  a  vented 
design  into  the  gas  cap 

65 

118 

27 

53 

There  is  no  disconnect 
where  the  fuel  transfer 
enters  the  fuel  pod 

Need  to  add  a  quick 
disconnect  mechanism  at 
the  pod  connection 

166 

177 

48 

11 

What  kind  of  feature 
for  changing  from 
water,  fuel,  or  ammo 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

201 

233 

24 

32 

Is  there  any  armor 
protection  required? 

No,  armor  is  not  an 
operational  requirement 

540 

593 

307 

53 

The  profile  of  the  tank 
is  too  high 

Lower  profile  of  the  fuel 
pod  to  reduce  the  trailer 
silouhette,  reduce 
vulnerability 

598 

659 

5 

61 

What  kind  of  feature 
for  changing  from 
water,  fuel,  or  ammo 

Need  to  have  more 
distinction  on  the  pod  as 
to  what  it  carries 

670 

874 

11 

204 

Team 

Experiment  Order 

Technology 

Subassembly 

T; 

Ei 

B 

n 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

The  trailer  needs  to  be 
protected 

Does  not  appear  to  be 
any  way  to  armor  the 
trailer  fuel  pod 

114 

432 

114 

318 

Profile  of  the  tank  is 
too  high 

Make  fuel  pod  longer 
and  lower  to  reduce 
vulnerability 

489 

505 

57 

16 

232 


Docs  the  fuel  pod  need 
separate  lifting  hooks? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

598 

622 

93 

24 

There  is  an  issue  when 
air  lifting  the  fuel  pod 
when  it  is  full,  as  far 
as  the  rigidity  of  the 
pod  (should  it  be 
flexible  or  rigid 

This  is  an  issue  that 
needs  to  be  addressed  in 
further  concept  work. 

644 

663 

22 

19 

The  fuel  pod  needs 
some  sort  of  venting 
system 

Provide  a  venting 
system  in  the  fuel  pod 
itself,  to  eliminate 
combustion 

706 

760 

43 

54 

Does  the  fuel  pod  need 
separate  lifting  hooks? 

Put  hooks  on  the  fuel 
pod  or  provide  some 
type  of  track  so  that  it 
can  be  removed  easily 
(slide  on  and  off) 

802 

834 

42 

32 

Team 

Experiment  Order 

Technology 

Subassembly 

t2 

E: 

A 

4 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

Ail  the  concept  has  is 
springs  and  one  axle, 
this  won’t  work 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

105 

108 

105 

3 

There  appears  to  be  no 
break  system 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

140 

185 

32 

45 

Only  one  axle  with 

600  gallons  on  top  of 
it 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

203 

260 

18 

57 

No  tail  lights  on  trailer 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

269 

293 

9 

24 

Still  concerned  about 
the  springs 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

320 

331 

27 

ll 

Back  to  600  gallon 
fuel  pod 

Need  a  two  axle  system 

336 

349 

5 

13 

Back  to  tail  lights  on 
trailer 

Need  to  put  a  set  of  rear 
lights  on  the  trailer 

356 

374 

7 

18 

I 

233 


I 


. 


Tank  assembly  is  short 

Make  it  longer,  but  this 
is  not  a  big  deal 

488 

547 

114 

59 

Can’t  tell  dynamically 
how  is  will  hold  up 

Need  some  dynamic 
modeling  conducted 

868 

908 

321 

40 

There  appears  to  be  no 
break  system 

Recap  -  Need  to  design  a 
braking  system  that  will 
brake  in  cooperation 
with  the  prime  mover 

970 

1008 

62 

38 

Only  one  axle  with 

600  gallons  on  top  of 
it 

Recap-Need  a  two  axle 
system  to  carry  weight 

1048 

1064 

40 

16 

No  tail  lights  on  trailer 

Recap-Nced  to  put  a  set 
of  rear  lights  on  the 
trailer 

1082 

1096 

18 

14 

All  the  concept  has  is 
springs  and  one  axle, 
this  won't  work 

Recap-Need  some  kind 
of  air  suspension  system 

1121 

1163 

25 

42 

Team 

Experiment  Order 

Technology 

Subassembly 

t2 

e3 

D 

3  1 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

Does  the  tow  pintle 
rotate? 

Never  Resolved 

105 

120 

105 

15 

Is  there  a  tow  hitch  on 
the  Ml? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

147 

180 

27 

33 

Needs  to  be  some 
landing  mechanism 

Incorporate  a  landing  leg 
into  the  design 

189 

197 

9 

8 

Is  there  a  tow  hitch  on 
the  Ml? 

Use  an  inventory  hitch 
system 

200 

221 

3 

21 

Needs  to  be  some 
landing  mechanism 

Incorporate  a  landing  leg 
into  the  design 

257 

332 

36 

75 

Problem  with  turning 
radius  when  turning 
the  trailer 

train  crew  how  to  drive 
when  trailer  in  tow 

347 

486 

15 

139 

Lifting  eyes  not  visible 

Need  eyehooks  with 
chain  spreaders  or  a 
flexible  lifting  belt 

643 

711 

157 

68 

Shorter  trailer  may  be 
better 

Need  a  better  turning 
radius 

811 

851 

100 

40 

234 


Is  the  quick  disconnect 
adequate 

Appears  to  be  OK 

1063 

1083 

212 

20 

Lifting  eyes  on  back 
end  of  the  trailer  seem 
as  if  it  would 
unbalance  the  trailer 

Extend  the  back  of  the 
trailer  out  a  little  bit  to 
stabilize  the  load  when 
airlifting 

1100 

1135 

17 

35 

Backend  of  the  trailer 
looks  too  light 

Extend  the  solid  metal 
sides  on  the  trailer  to  the 
back  end 

1280 

1313 

145 

33 

Team 

Experiment  Order 

Technology 

Subassembly 

t2 

E4 

c 

i 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 

(Sec) 

Times  to 
Resolve 
(Sec) 

The  piping  is  too  rigid 

Need  a  flexible  hose 

71 

96 

71 

25 

Why  fill  fuel  from  the 
top? 

Use  a  bottom  filler  cap 

173 

197 

77 

24 

Where  should  the 
bottom  cap  be  place? 

Use  a  hose  from  bottom 
and  boom  it  up  like  a 
flexible  joystick 

247 

267 

50 

20 

There  is  a  lot  of  heat 
coming  out  of  the  Ml 
exhaust  system 

This  needs  to  be 
addressed  with  some 
sort  of  heat  shielding 
mechanism 

376 

386 

109 

10 

When  the  turret 
rotates,  the  gun  tube 
will  rip  off  the  fuel 
transfer  line 

The  hose  needs  to  come 
out  of  the  bottom 

394 

411 

8 

17 

Is  there  quick 
disconnects  at  tank 
side  and  pod  side? 

Need  to  have  flexible, 
rotatable  disconnects  at 
both  ends 

505 

517 

94 

12 

Is  fuel  sucked  out  or 
pumped  out? 

Include  a  pony  engine 
on  the  trailer  to  pump 
the  fuel  if  it’s  not  using  a 
suction  device 

528 

650 

11 

122 

Is  it  constantly  fueling 
or  not? 

Provide  a  sensor  on  the 
Ml  tank  so  that  it  knows 
the  fuel  level  and  when 
it  is  completely  out 

733 

794 

83 

61 

235 


Should  a  remote 
control  be  placed  on 
the  Ml  tank  to  turn  the 
pump  on  or  off? 

SSS  vs.  ease  of 
installation  vs.  automatic 
continuous  fueling 

841 

922 

47 

81 

Does  the  driver  need 
two  fuel  gauges? 

No,  the  drivers  primary 
concern  is  the  fuel  in  the 
Ml  and  not  the  fuel  pod 

952 

1044 

30 

92 

Team 

Experiment  Order 

Technology 

Subassembly 

Tj 

E, 

c 

3 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

Are  there  any  airlift 
holds  on  the  trailer  and 
are  they  enough? 

Located  4  hold  points, 
this  seems  sufficient 

113 

164 

113 

51 

Is  the  pintle  on  the 
tank  the  normal  M 1 
pintle? 

not  resolved 

190 

410 

26 

220 

How  does  the  hitch 
work? 

Eyelet  on  the  trailer 
hooks  onto  the  lower 
portion  of  the  pintle 

420 

468 

10 

48 

Fluctuations  in  terrain 
are  going  to  cause 
problems  in  the 
hookup  as  far  as 
rotation,  bouncing  off 
the  hitch 

Need  a  pivotable  hitch 
and  one  that  clamps  and 
locks  down 

470 

480 

2 

10 

There  are  no  safety 
chains 

not  resolved 

489 

665 

9 

176 

Is  the  hitch  position 
locked 

Not  resolved 

678 

970 

13 

292 

How  do  the  pintle  jaws 
work? 

Need  to  look  at  the 
existing  Ml  pintle  and 
compare 

1024 

1159 

54 

135 

Eyelet  on  the  trailer 
isn’t  bolted  to  the 
frame 

Bolt  or  weld  it  to  the 
frame 

1284 

1298 

125 

14 

How  is  the  trailer 
leveled  to  the  pintle 
height? 

Need  something  to  level 
the  trailer  to  the  height 
of  the  pintle  in  order  to 
connect  it  easily 

1360 

1368 

62 

8 

236 


Team 

Experiment  Order 

Technology 

Subassembly 

T3 

e2 

D 

i 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

There  is  going  to  be  a 
fatigue  problem  with 
the  rigid  z*curve  of  the 
fuel  transfer  system 

Replace  rigid  pieces 
with  flexible  tubing 

25 

56 

25 

31 

How  is  attached  to  the 
tank? 

No  resolution 

59 

90 

3 

31 

Where  is  the  fuel 
pump?  How  is  the  fuel 
pumped  into  the  M 1 
tank? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

94 

120 

4 

26 

Why  is  the  fuel  line 
coming  out  of  the  top? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

124 

183 

4 

59 

Where  are  the 
electrical  connections 
for  the  trailer 
subassemblies? 

Provide  an  electrical 
harness  of  some  kind  to 
the  trailer  system 

216 

386 

96 

170 

Where  is  the  fuel 
pump?  How  is  the  fuel 
pumped  into  the  Ml 
tank? 

Need  something  in  the 
Ml  to  suck  or  suction 
the  fuel  out  or  even  a 
pump  in  the  fuel  tank 
itself 

390 

428 

4 

38 

Why  is  the  fuel  line 
coming  out  of  the  top? 

Locate  fuel  line  towards 
the  middle  or  bottom  of 
tank 

440 

526 

12 

86 

The  manhole  cover  in 
the  fuel  pod  is  too 
small  for  conducting 
any  maintenance  on 
the  pod 

Enlarge  the  manhole 
cover 

620 

638 

94 

18 

How  is  the  fuel  filtered 
from  the  trailer  to  the 
Ml  tank? 

Put  a  filter  in  the  fuel 
line  of  the  tank  for 
filtering  all  the  fuel 

706 

716 

68 

10 

Diameter  of  the  fuel 
transfer  rod  is  too 
small 

Widen  the  diameter  of 
the  hose  feeding  the  tank 

768 

798 

52 

30 

237 


Team 

Experiment  Order 

Technology 

Subassembly 

T3 

e3 

A 

2 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

How  is  the  pintle 
attached  to  the  trailer? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

63 

240 

63 

177 

How  is  the  fuel  pod 
removed? 

Provide  a  mounting 
feature  on  the  fuel  pod 
itself 

288 

338 

48 

50 

How  is  the  pintle 
attached  to  the  trailer? 

Need  a  quick  method  for 
attaching  and  detaching 
the  trailer  from  the 
pintle 

345 

379 

7 

34 

Logistics  is  a  problem 
when  changing  from 
fuel  to  water. 

Probably  will  not  do  this 
during  batde  this  will  be 
done  easily  in  non-battle 
conditions 

485 

586 

106" 

101 

Does  the  fuel  pod  need 
to  be  a  lower  profile? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

587 

615 

1 

28 

How  are  gas  fumes  in 
the  pod  taken  care  of? 

Need  baffles  installed  on 
the  fuel  pod 

628 

648 

13 

20 

The  fuel  pod  is  too 
much  design  in  the 
shape  of  a  fuel  tank 
and  thus  making  it 
more  vulnerable  for 
attack 

Redesign  the  pod  to 
resemble  something  else 
that  the  Ml  normally 
tows 

819 

880 

171 

61 

Does  the  fuel  pod  need 
to  be  a  lower  profile? 

No  profile  change  is 
really  needed,  it  is  not 
higher  than  the  Ml  tank 

900 

940 

20 

40 

Team 

Experiment  Order 

Technology 

Subassembly 

T3 

B 

4 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

Do  we  want  wheels 
with  the  rubber  tracks? 

Keep  the  concept  as  it  is, 
it  should  work 

36 

70 

36 

34 

238 


Use  of  leaf  springs 
only,  is  this  adequate? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

120 

148 

50 

28 

Detail  of  model 
doesn’t  show  how 
spring  is  attached  to 
the  axle 

Need  more  detail  to 
determine  how  they  are 
connected 

288 

317 

140 

29 

Use  of  leaf  springs 
only,  is  this  adequate? 

Some  type  of  shock 
absorption  is  required 

320 

374 

3 

54 

How  is  the  track  taken 
off  or  replaced? 

Provide  some  type  of 
mechanical  system  to 
aid  in  the  removal  of  the 
track 

458 

498 

84 

40 

Mud  splashing  will  be 
a  problem 

Provide  some  type  of 
fender  system  to  keep 
the  mud  splashing 
contained 

540 

556 

42 

16 

Team 

Experiment  Order 

Technology 

Subassembly 

T4 

E,  , 

D 

4 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

Why  do  they  have 
tracks  on  the  trailer? 

Study  needs  to  be  done 
on  whether  the  track 
possibility  is  required 

24 

279 

24 

255 

Why  tandem  wheels 
on  each  side? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

293 

316 

14 

23 

No  oscillation  ability 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

504 

810 

188 

306 

Height  of  trailer,  is  it 

O.K.? 

Good  height  from  the 
ground  clearance  of 
ground  obstacles 

840 

862 

30 

22 

Spring  hangers  look 
flimsy 

Re-look  at  the  design  of 
the  hangers,  they  need  to 
be  strengthened 

888 

960 

26 

72 

Why  tandem  wheels 
on  each  side? 

Look  at  using  an  in 
stock  large  wheel  that 
are  being  used  on  other 
vehicles  (one  vs.  two) 

970 

982 

10 

12 

What  holds  the  track 
on? 

No  sign  of  a  track  guide, 
one  needs  to  be  inserted 

1203 

1228 

221 

25 

239 


Tires  are  very  wide 

Relief  of  ground 
pressure  a  possibility, 
need  to  conduct  an 
analysis 

1290 

1374 

62 

34 

Axle  meeting  wheel 
bar,  no  pivot  on  bar 

How  are  bars  attached  to 
the  wheel,  need 
something  to  attach 
them 

1357 

1505 

33 

148 

How  are  things 
attached? 

Need  more  detail  in  the 
design 

1548 

1610 

43 

62 

Will  track  wear  be  an 
issue? 

Conduct  a  durability 
analysis. 

1622 

1670 

12 

48 

Team 

Experiment  Order 

Technology 

Subassembly 

Tj 

E  2 

c 

2 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

Profile  of  pod  is  too 
high 

Reduce  the  height, 
spread  out  over  the 
length  of  the  frame 
making  it  more  stable 

46 

64 

46 

18 

What  is  the  tank  made 
of? 

Need  to  determine  a 
suitable  material 

142 

156 

78 

14 

Why  not  a  fuel 
bladder,  why  use  a 
trailer  at  all? 

A  bladder  directly  on  the 
tank  may  be  a  better 
solution 

167 

208 

11 

41 

Is  there  a  pump? 

Needs  to  be  a  pumping 
system 

226 

248 

18 

22 

Attachment  to  trailer, 
stress  points 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

316 

328 

68 

12 

How  is  it  vented? 

Need  baffles  to  vent  it 

361 

410 

33 

49 

How  do  determine  the 
fuel  level? 

Place  a  fuel  gauge  on  the 
pod 

468 

471 

58 

3 

Need  to  drain  the  fuel 

Add  a  drain  to  the  fuel 
pod 

478 

490 

7 

12 

Need  identification  of 
what  is  in  the  pod 

Add  ID  plate 

708 

721 

218 

13 

Attachment  to  trailer, 
stress  points 

Need  to  place  a  damper 
between  the  fuel  pod 
and  where  it  rests  on  the 
trailer 

748 

791 

27 

43 

240 


Lifting  eyes  or  crack 
for  interchangeability 

Need  a  way  to  change 
the  tank,  lift  off  the  pods 

899 

929 

108 

30 

Overpressure  valve  is 
not  apparent 

Make  sure  to  include  an 
overpressure  valve 

948 

969 

19 

21 

Team 

Experiment  Order 

Technology 

Subassembly 

t4 

B 

I 

Problem/Issues 

Resolution 

Detection 
Time  (Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 

(Sec) 

Times  to 
Resolve 
(Sec) 

Hard  piping/line  size 
flow  not  good 

use  a  flexible  hose  and 
increase  the  diameter  of 
the  line 

145 

158 

145 

13 

Engine  heat  exhaust 
will  be  a  problem 

Use  thermal  hosing 

190 

197 

32 

7 

How  often  running  the 
transfer  system 

Felt  all  fuel  should  be 
transferred  and  then  the 
trailer  should  be 
jettisoned 

252 

336 

55 

84 

How  do  quick 
disconnects  work 
remotely? 

Did  not  resolve,  but 
decided  to  move  to  next 
problem 

356 

420 

20 

64 

Square  transfer  pipe? 

Use  the  flexible  hose 
method 

489 

511 

69 

22 

How  do  quick 
disconnects  work 
remotely? 

Possibility  of  using  a 
pressure  disconnect  of 
some  type 

677 

870 

166 

193 

Corrosion  problems 

Need  to  use  materials 
that  are  non-corrosive 

878 

895 

8 

17 

Team 

Experiment  Order 

Technology 

Subassembly 

t4 

Ea 

A 

3 

Problem/Issues 

Resolution 

Detection  Time 
(Sec) 

Resolution 
Time  (Sec) 

Times  to 
Detect 
(Sec) 

Times  to 
Resolve 
(Sec) 

Where  the  pintle  bar 
meets  the  trailer  frame, 
tension  and  sheer  will 
cause  it  to  fail  and  it 
may  snap. 

Lengthen  the  bar  into 
the  eyelet  and  fasten 
it  down 

25 

160 

25 

135 

241 


Eyelet  is  too  small 

Needs  to  be 
reworked 

170 

240 

10 

70 

No  auto  disconnect 

Need  a  remote 
disconnect,  solenoid 
of  some  type  may  be 
the  solution 

251 

275 

11 

24 

No  pincer  to  drop  the 
pintle 

Need  something  that 
disconnects  and  lets 
the  pintle  loose  from 
the  eyelet 

449 

548 

174 

99 

Pintle  is  too  high  on 
the  back  of  the  tank 

Lower  the  pintle  on 
the  tank 

571 

604 

23 

33 

Should  the  entire  are 
flex? 

Yes,  or  cracking  will 
occur  in  the  towing 
structure  if  there  is 
no  flex 

668 

721 

64 

53 

242 


Appendix  K 


Number  of  Design  Error  Detected 
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Number  of  Design  Errors  Detected  for  Each  Treatment 


Team 

Experiment 

Order 

Technology 

Subassembly 

Errors 

Ti 

Ei 

A 

1 

3 

T, 

e2 

B 

3 

4 

Ti 

e3 

C 

4 

7 

Ti 

E4 

D 

2 

5 

t2 

E, 

B 

2 

5 

t2 

e2 

A 

4 

6 

t2 

e3 

D 

3 

9 

t2 

E4 

C 

1 

10 

t3 

Ei 

C 

3 

9 

t3 

e2 

D 

1 

8 

t3 

e3 

A 

2 

6 

t3 

E4 

B 

4 

5 

t4 

Ei 

D 

4 

10 

t4 

e2 

C 

2 

11 

t4 

e3 

B 

1 

6 

t4 

E4 

A 

3 

6 
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Appendix  L 


Design  Problem  Detection  Time 
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Mean  and  Standard  Deviation  for  Each  Treatment  on  Detection 


Times  (Seconds) 


Group 

Experiment  Technology  Subassembly 
Order 

Mean 

SD 

N 

T, 

Ei 

A 

1 

42.75 

46.11 

4 

Ti 

e2 

B 

3 

74.2 

44.09 

5 

Ti 

e3 

C 

4 

96.29 

119.76 

7 

T, 

Ea 

D 

2 

64.57 

107.79 

7 

t2 

Ei 

B 

2 

61.83 

34.79 

6 

t2 

e2 

A 

4 

60.23 

85.89 

13 

t2 

e3 

D 

3 

75.09 

72.78 

11 

t2 

e4 

C 

1 

58 

34.529 

10 

t3 

Ei 

C 

3 

46 

46.29 

9 

t3 

e2 

D 

1 

36.2 

38.15 

10 

t3 

e3 

A 

2 

53.63 

58.99 

8 

t3 

E4 

B 

4 

59.17 

47.373 

6 

t4 

Ei 

D 

4 

55.25 

73.21 1 

11 

t4 

e2 

C 

2 

57.58 

58.97 

12 

t4 

e3 

B 

1 

70.71 

58.52 

7 

t4 

E4 

A 

3 

51.17 

63.3 

6 

246 


Appendix  M 


Data  for  Design  Problem  Resolution  Time 
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Mean  and  Standard  Deviation  for  Each  Treatment  on  the  average  time  to  resolve  design 

errors  (Seconds) 


Group 

Experiment  Technology  Subassembly  Mean 
Order 

SD 

N 

T, 

E, 

A 

1 

75.5 

71.06 

4 

T, 

e2 

B 

3 

174.2 

138.93 

5 

T, 

e3 

C 

4 

133.71 

149.38 

7 

Ti 

E4 

D 

2 

60.29 

66.74 

7 

t2 

Ei 

B 

2 

77.17 

118.77 

6 

t2 

e2 

A 

4 

29.23 

18.50 

13 

t2 

e3 

D 

3 

44.27 

37.61 

11 

t2 

e4 

C 

1 

46.4 

39.77 

10 

t3 

Ei 

C 

3 

106 

104.28 

9 

t3 

e2 

D 

1 

49.9 

47.43 

10 

t3 

e3 

A 

2 

63.88 

52.17 

8 

t3 

E4 

B 

4 

33.5 

12.80 

6 

t4 

E, 

D 

4 

91.55 

101.31 

11 

T4 

e2 

C 

2 

23.17 

14.47 

12 

t4 

e3 

B 

1 

57.14 

66.5 

7 

t4 

e4 

A 

3 

69 

42.03 

6 

248 


Appendix  N 


Participant  Comments  from  VE  Assessment  Survey  Two 
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Technology  A,  HMD 


Provide  comments  regarding  the  effect  of  this  tool  to  perform  concept  design 
review  tasks,  the  quality  of  the  technology  as  it  applies  to  product  design  and  your 
overall  reaction  to  the  technology,  and  any  comment  regarding  your  physical  or 
emotional  state  after  using  the  technology. 

Should  be  able  to  zoom  in  an  out  of  model  components  in  order  to  examine  them 
more  thoroughly. 

The  HMD  is  not  useful  for  the  actual  design  because  the  design  can  not  be  modified. 

HMD  allows  for  easy  maneuvering  around  the  concept. 

You  are  able  to  see  the  design  in  3-D  and  really  concentrate  on  a  specific  area  of 
the  design. 

Seems  good  because  it  raised  many  questions  regarding  the  concept  in  a  short 
time. 

I  feel  that  this  technology  greatly  improves  the  overall  product  design. 

Easy  to  learn  and  utilize. 

If  properly  developed  this  tool  has  the  potential  to  enhance  the  design  process. 

However,  in  its  current  state  it  has  little  use. 

Particularly  difficult  in  team  reviews  since  you  can  not  point  at  things. 

The  person  wearing  the  device  is  not  part  of  the  group;  perhaps  each  group 
member  should  be  wearing  a  device. 

No  eye  contact  when  wearing  the  device,  so  you  can’t  understand  what  others  are 
referring  to. 

Locking  in  on  a  view  is  hard  to  due. 

Easiest  device  to  use. 

Clear  picture  and  ease  of  use  are  positive  attributes 

Is  there  any  aspect  of  the  visualization  tool  that  could  be  improved  to 
enhance  the  concept  design  review  activity? 

Improve  component  level  detail  and  exploration. 

Smooth  the  movement  of  the  model. 

Improve  resolution. 

If  the  tool  really  allowed  you  to  touch  the  object  under  study  it  would  be  very 
useful. 

Have  the  ability  to  lock  in  a  view. 
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Technology  B,  BOOM 


Provide  comments  regarding  the  effect  of  this  tool  to  perform  concept  design 
review  tasks,  the  quality  of  the  technology  as  it  applies  to  product  design  and  your 
overall  reaction  to  the  technology,  and  any  comment  regarding  your  physical  or 
emotional  state  after  using  the  technology. 

The  low  image  resolution  makes  it  difficult  to  determine  the  details  of  the 
concept;  it  was  easier  to  see  it  on  the  monitor. 

It  is  difficult  to  maneuver  into  the  viewing  angle  you  desire. 

The  image  seems  jumpy. 

Very  poor  image. 

Difficult  to  move  around  the  model. 

Horizontal  movement  is  ok,  but  vertical  movement  is  difficult. 

Image  very  grainy. 

Some  feelings  of  nausea. 

More  stimulating  than  looking  at  drawings,  it  could  make  work  more  interesting. 
It  does  not  appear  to  be  a  viable  tool  since  the  user  must  stand  up  in  a  relatively 
uncomfortable  posture. 

Can  not  quickly  view  what  you  want  to  see. 

The  BOOM  was  hindersome;  once  we  got  the  object  in  view  we  stepped  away 
and  discussed. 

No  physical  or  emotional  effects. 

Mouse  is  better  to  accomplish  same  task  of  moving. 

Easy  to  learn. 

Without  the  external  monitor  the  team  would  not  be  able  to  communicate  or  see 
the  same  item  at  the  same  instance. 

Very  useful,  freedom  to  move  is  easy  and  concise. 

User  doesn’t  get  frustrated  trying  to  aim  where  they  would  like  to  be  located. 

Easy  flowing. 

My  eyes  were  unaffected  by  the  technology,  the  updates  were  slow  enough  not  to 
make  me  dizzy. 

Device  doesn’t  lend  itself  for  office  use. 

I  could  not  use  on  a  daily  basis. 

Felt  that  it  required  too  much  movement  of  the  users'  head  and  I  think  that  would 
be  uncomfortable. 

Equipment  is  cumbersome. 

Is  there  any  aspect  of  the  visualization  tool  that  could  be  improved  to 
enhance  the  concept  design  review  activity? 

Image  quality. 

Improve  vertical  movement  for  ease  of  operation. 

Keep  bumping  into  BOOM  arm. 
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The  structure  of  the  BOOM  causes  the  user  problems  because  one  keeps  hitting  it 
with  their  head  when  turning  at  certain  times. 

Attach  it  to  a  swivel  chair  so  the  user  can  sit  down  while  operating  it. 

Technology  C.  Stereoscopic  Glasses 

Provide  comments  regarding  the  effect  of  this  tool  to  perform  concept  design 
review  tasks,  the  quality  of  the  technology  as  it  applies  to  product  design  and  your 
overall  reaction  to  the  technology,  and  any  comment  regarding  your  physical  or 
emotional  state  after  using  the  technology. 

Difficult  to  use  the  mouse  to  maneuver  to  desired  viewing  angle. 

Mouse  functions  difficult. 

Model  too  slow. 

Need  to  focus  better,  resolution. 

The  glasses  did  not  assist  in  my  concept  review  using  the  monitor. 

Glasses  didn’t  seem  to  make  much  difference  to  me. 

Glasses  didn’t  seem  to  help  in  the  discussion. 

A  good  tool  if  used  in  accordance  with  a  2-D  drawing. 

Had  a  problem  with  “ghost”  images. 

Difficulty  in  focusing  on  the  solid  model. 

Developed  a  bit  of  eyestrain  trying  to  focus. 

The  current  state  of  the  technology  does  not  appear  to  be  efficient  tool  to  evaluate 
design  problems;  it  may  be  useful  to  look  at  the  overall  design  after  the 
problems  have  been  resolved. 

Problem  is  getting  in  close. 

The  benefit  of  3-D  glasses  did  not  add  to  enhancing  our  ability  to  visualize  our 
sample.  Maybe  this  is  more  suited  for  more  complex  or  detailed  designs. 
More  eye  demanding  than  a  simple  monitor  does. 

Wouldn’t  want  to  use  this  technology  for  a  long  design  review. 

Glasses  are  hindersome  at  times. 

Is  there  any  aspect  of  the  visualization  tool  that  could  be  improved  to 
enhance  the  concept  design  review  activity? 

Make  it  easier  to  change  viewing  orientation. 

Actually  get  rid  of  the  glasses. 

Difficulty  to  position  at  the  correct  spot  to  get  the  best  3-D. 
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Technology  D,  3-D  Monoscopic  CRT  Monitor 


Provide  comments  regarding  the  effect  of  this  tool  to  perform  concept  design 
review  tasks,  the  quality  of  the  technology  as  it  applies  to  product  design  and  your 
overall  reaction  to  the  technology,  and  any  comment  regarding  your  physical  or 
emotional  state  after  using  the  technology. 

Difficult  using  the  mouse  to  maneuver  to  desired  viewing  angle. 

The  stereo  effect  would  be  more  useful  for  more  complex  models. 

Less  tasking  on  one  person,  allowing  the  group  to  interact  more. 

Slow  updating  causes  group  to  lose  focus. 

The  mouse  was  the  easiest  device  to  use. 

Helped  me  pinpoint  design  problems. 

Helps  other  people  visualize  what  the  engineer  is  trying  to  develop. 

Provides  a  better  understanding  of  the  concept. 

The  tool  was  more  difficult  to  use  due  to  inexperience,  than  using  drawings  and 
appropriate  design  parameters.  However,  the  future  designer  may  use  this 
type  of  technology  in  conjunction  with  other  equipment  to  optimize  the 
design  process. 

Was  easy  to  move  around  and  look  at  different  views  of  a  picture. 

It  was  as  familiar  as  using  your  PC. 

No  physical  or  emotional  effects. 

One  can  see  how  subsystems  interface  with  each  other. 

The  quality  of  the  graphics  was  crisp,  clear  and  allowed  for  any  flaws  to  be 
detected  easily. 

Great  tool  for  assisting  in  the  design  process. 

Networking  the  design  review  through  a  series  of  screens  integrating  the  engineer, 
logistician,  maintenance  and  cost  estimator  will  result  in  optimal  designs 
that  are  both  user  friendly  in  terms  of  operation,  maintaining,  and  cost 
effective. 

Command  controls  are  slow  and  awkward  to  use  (at  least  initially)  this  can 

become  tiresome  and  can  exuberate  an  already  difficult  task  like  a  design 
review. 

Is  there  any  aspect  of  the  visualization  tool  that  could  be  improved  to 
enhance  the  concept  design  review  activity? 

Make  it  easier  to  change  views. 

Would  like  to  make  design  changes  on  the  spot  (cut,  paste,  and  delete). 

A  large  monitor  screen  with  more  details  including  both  the  electrical  and 
mechanical  portions  of  the  system. 
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